-- Logs begin at Fri 2022-08-05 18:45:44 UTC, end at Fri 2022-08-05 19:56:45 UTC. -- Aug 05 19:48:05.545668 dv-cinder-852208-2-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Aug 05 19:48:05.561469 dv-cinder-852208-2-iscsi devstack@keystone.service[80352]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Aug 05 19:48:05.563914 dv-cinder-852208-2-iscsi devstack@keystone.service[80352]: *** Starting uWSGI 2.0.18-debian (64bit) on [Fri Aug 5 19:48:05 2022] *** Aug 05 19:48:05.563914 dv-cinder-852208-2-iscsi devstack@keystone.service[80352]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Aug 05 19:48:05.563914 dv-cinder-852208-2-iscsi devstack@keystone.service[80352]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Aug 05 19:48:05.563914 dv-cinder-852208-2-iscsi devstack@keystone.service[80352]: nodename: dv-cinder-852208-2-iscsi Aug 05 19:48:05.563914 dv-cinder-852208-2-iscsi devstack@keystone.service[80352]: machine: x86_64 Aug 05 19:48:05.563914 dv-cinder-852208-2-iscsi devstack@keystone.service[80352]: clock source: unix Aug 05 19:48:05.563914 dv-cinder-852208-2-iscsi devstack@keystone.service[80352]: pcre jit disabled Aug 05 19:48:05.563914 dv-cinder-852208-2-iscsi devstack@keystone.service[80352]: detected number of CPU cores: 4 Aug 05 19:48:05.563914 dv-cinder-852208-2-iscsi devstack@keystone.service[80352]: current working directory: / Aug 05 19:48:05.563914 dv-cinder-852208-2-iscsi devstack@keystone.service[80352]: detected binary path: /usr/bin/uwsgi-core Aug 05 19:48:05.565549 dv-cinder-852208-2-iscsi devstack@keystone.service[80352]: your processes number limit is 47680 Aug 05 19:48:05.565549 dv-cinder-852208-2-iscsi devstack@keystone.service[80352]: your memory page size is 4096 bytes Aug 05 19:48:05.565549 dv-cinder-852208-2-iscsi devstack@keystone.service[80352]: detected max file descriptor number: 2048 Aug 05 19:48:05.565549 dv-cinder-852208-2-iscsi devstack@keystone.service[80352]: lock engine: pthread robust mutexes Aug 05 19:48:05.565549 dv-cinder-852208-2-iscsi devstack@keystone.service[80352]: thunder lock: enabled Aug 05 19:48:05.565549 dv-cinder-852208-2-iscsi devstack@keystone.service[80352]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Aug 05 19:48:05.565549 dv-cinder-852208-2-iscsi devstack@keystone.service[80352]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Aug 05 19:48:05.605676 dv-cinder-852208-2-iscsi devstack@keystone.service[80352]: Python main interpreter initialized at 0x55bb4a6289c0 Aug 05 19:48:05.605676 dv-cinder-852208-2-iscsi devstack@keystone.service[80352]: python threads support enabled Aug 05 19:48:05.605676 dv-cinder-852208-2-iscsi devstack@keystone.service[80352]: your server socket listen backlog is limited to 100 connections Aug 05 19:48:05.605676 dv-cinder-852208-2-iscsi devstack@keystone.service[80352]: your mercy for graceful operations on workers is 90 seconds Aug 05 19:48:05.606068 dv-cinder-852208-2-iscsi devstack@keystone.service[80352]: mapped 403077 bytes (393 KB) for 2 cores Aug 05 19:48:05.606126 dv-cinder-852208-2-iscsi devstack@keystone.service[80352]: *** Operational MODE: preforking *** Aug 05 19:48:05.606403 dv-cinder-852208-2-iscsi devstack@keystone.service[80352]: *** uWSGI is running in multiple interpreter mode *** Aug 05 19:48:05.606403 dv-cinder-852208-2-iscsi devstack@keystone.service[80352]: spawned uWSGI master process (pid: 80352) Aug 05 19:48:05.606650 dv-cinder-852208-2-iscsi systemd[1]: Started Devstack devstack@keystone.service. Aug 05 19:48:05.609200 dv-cinder-852208-2-iscsi devstack@keystone.service[80352]: spawned uWSGI worker 1 (pid: 80363, cores: 1) Aug 05 19:48:05.609200 dv-cinder-852208-2-iscsi devstack@keystone.service[80352]: spawned uWSGI worker 2 (pid: 80364, cores: 1) Aug 05 19:48:05.609200 dv-cinder-852208-2-iscsi devstack@keystone.service[80352]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Aug 05 19:48:11.431893 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 05 19:48:11.431893 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: from cryptography.utils import int_from_bytes Aug 05 19:48:11.432782 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 05 19:48:11.432782 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: from cryptography.utils import int_from_bytes Aug 05 19:48:11.516250 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 05 19:48:11.516490 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 05 19:48:11.516718 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] command line args: [] {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 05 19:48:11.517022 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 05 19:48:11.517201 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] ================================================================================ {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 05 19:48:11.517984 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] admin_token = **** {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 19:48:11.518230 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] config_dir = [] {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 19:48:11.518419 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 19:48:11.518647 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] config_source = [] {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 19:48:11.518872 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] debug = True {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 19:48:11.519077 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 19:48:11.519522 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 19:48:11.519769 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 19:48:11.519987 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 19:48:11.520242 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 19:48:11.520480 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] list_limit = None {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 19:48:11.520659 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] log_config_append = None {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 19:48:11.520922 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 19:48:11.521048 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] log_dir = None {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 19:48:11.521274 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] log_file = None {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 19:48:11.521498 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 19:48:11.521749 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 19:48:11.521998 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 19:48:11.522261 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 19:48:11.522425 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 19:48:11.522595 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 19:48:11.522854 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 19:48:11.523082 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 19:48:11.523318 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 19:48:11.523554 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 19:48:11.523784 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 19:48:11.524021 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 19:48:11.524274 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 19:48:11.524800 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 19:48:11.525079 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 19:48:11.525598 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.27/identity {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 19:48:11.525778 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] publish_errors = False {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 19:48:11.526075 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 19:48:11.526247 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 19:48:11.526426 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 19:48:11.526670 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 19:48:11.526922 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 19:48:11.527148 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] standard_threads = False {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 19:48:11.527377 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 19:48:11.527621 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 19:48:11.527840 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 19:48:11.528014 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] use_journal = False {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 19:48:11.528195 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] use_json = False {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 19:48:11.528368 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] use_stderr = False {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 19:48:11.528560 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] use_syslog = False {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 19:48:11.528790 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 19:48:11.529053 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.529290 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.529548 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.529786 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.530034 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.530276 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.530523 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.530759 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.531007 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.531252 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.531510 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.531742 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.531994 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.532223 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.532467 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.532739 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.533053 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.533815 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.534069 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.534318 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.534554 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.534808 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.535073 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.535215 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 05 19:48:11.535215 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: from cryptography.utils import int_from_bytes Aug 05 19:48:11.535986 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.535986 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.536218 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 05 19:48:11.536218 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: from cryptography.utils import int_from_bytes Aug 05 19:48:11.536425 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.536425 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.536425 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.536425 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.536942 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.536942 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.537172 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.537381 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.537625 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.537872 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.538111 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.538359 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.538589 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.538823 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.539043 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.539275 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.539503 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.539734 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.539963 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.540201 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.540441 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.540699 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.540937 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.541164 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.541398 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.541630 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.541878 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.542106 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.542347 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.542583 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.542820 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.543051 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.543286 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.543517 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.543764 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.544042 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.544343 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.544677 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.544999 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.545296 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.545592 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.545886 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.546188 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.546520 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.546822 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.547131 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.547445 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.547738 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.548030 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.548326 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.548700 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.549015 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.549309 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.549620 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.549915 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.550212 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.550504 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.550804 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.551097 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.551424 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.551729 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.552029 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.552331 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.552674 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.552985 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.553288 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.553593 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.553887 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.554186 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.554481 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.554785 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.555083 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.555393 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.555703 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.556011 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.556316 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.556671 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.556988 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.557309 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.557604 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] auth.external = None {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.557912 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.558382 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.558673 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.558968 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] auth.password = None {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.559262 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] auth.token = None {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.559564 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.559866 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.560172 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.560467 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.560836 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.561151 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.561445 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.561741 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] credential.caching = True {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.562048 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.562381 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.562694 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.562999 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.563314 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.563611 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.563939 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.564352 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.564680 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.564994 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.565340 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.565648 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.565974 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.566274 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.566581 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.566862 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] federation.caching = True {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.567154 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.567445 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.567743 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.568017 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.568298 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.568638 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.568946 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.569235 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.569541 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.569830 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.570156 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.570435 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] identity.caching = True {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.570739 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.571028 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.571317 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.571603 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.571898 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.572181 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.572532 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.572866 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.573231 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.573862 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.574032 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.574378 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.574722 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.575083 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.575486 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.575851 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.576189 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.576588 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.576951 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.577288 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.577610 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.577939 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.578315 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.578652 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.578992 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.579328 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.579666 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.580522 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.580875 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.581224 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.581550 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.581885 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.582237 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.582568 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.582900 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.583217 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.583548 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.583886 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.584221 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.584725 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.585060 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.585402 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.585747 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.586078 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.586399 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.586734 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.587071 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.587397 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.587716 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 05 19:48:11.587894 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.588039 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 05 19:48:11.588175 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.588309 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] command line args: [] {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 05 19:48:11.588443 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 05 19:48:11.588615 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] ldap.user = None {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.588766 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] ================================================================================ {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 05 19:48:11.588907 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] admin_token = **** {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 19:48:11.589035 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.589167 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] config_dir = [] {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 19:48:11.589342 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 19:48:11.589478 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.589628 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] config_source = [] {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 19:48:11.589757 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] debug = True {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 19:48:11.589885 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.590025 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 19:48:11.590190 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 19:48:11.590336 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.590467 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 19:48:11.590594 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 19:48:11.590723 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.590853 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 19:48:11.591015 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] list_limit = None {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 19:48:11.591168 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.591300 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] log_config_append = None {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 19:48:11.591426 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 19:48:11.591561 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.591693 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] log_dir = None {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 19:48:11.591693 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] log_file = None {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 19:48:11.591944 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.592076 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 19:48:11.592206 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 19:48:11.592334 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.592465 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 19:48:11.592636 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 19:48:11.592784 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.592926 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 19:48:11.593058 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 19:48:11.593187 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.593333 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 19:48:11.593476 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 19:48:11.593617 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.593750 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 19:48:11.593908 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 19:48:11.594042 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.594173 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 19:48:11.594373 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 19:48:11.594533 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.594666 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 19:48:11.594795 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.595259 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 19:48:11.595259 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 19:48:11.595539 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.596294 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.27/identity {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 19:48:11.596541 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.596742 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] publish_errors = False {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 19:48:11.596942 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 19:48:11.597159 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.597357 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 19:48:11.597357 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 19:48:11.597720 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.598669 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 19:48:11.598669 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 19:48:11.599112 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.599330 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] standard_threads = False {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 19:48:11.599544 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.599764 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 19:48:11.599764 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 19:48:11.600183 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.600450 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 19:48:11.600450 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] use_journal = False {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 19:48:11.600908 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.601122 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] use_json = False {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 19:48:11.601122 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] use_stderr = False {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 19:48:11.601122 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] use_syslog = False {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 19:48:11.601713 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.601940 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 05 19:48:11.602166 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.602390 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.602390 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.602795 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.603009 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.603009 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.603425 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.603758 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.603758 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.604161 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.604357 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.604542 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.604698 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.604698 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.604952 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.605086 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.605086 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.605325 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.605469 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.605614 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.605744 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.605744 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.605963 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.606078 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.606078 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.606302 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.606419 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.606531 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.606647 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.606772 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.606990 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.607126 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] resource.caching = True {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.607269 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.607269 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.607509 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.607641 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.607641 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.607909 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.608040 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.608170 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.608298 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.608298 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.608576 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.608576 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.608817 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.608946 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.609075 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.609205 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.609347 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.609347 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.609586 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.609721 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.609850 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.609978 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.609978 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.610230 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] role.caching = True {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.610367 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.610367 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.610614 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] role.driver = sql {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.610744 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.610889 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.611051 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.611051 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.611290 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.611419 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.611419 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.611668 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.611801 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.611932 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.612063 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.612063 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.612327 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.612460 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.612460 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.612728 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.612858 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.612858 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.613213 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.613348 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.613348 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.613620 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.613753 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.613882 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.614025 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.614025 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.614248 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.614391 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.614391 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.614627 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.614769 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.614904 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.615034 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.615034 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.615271 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.615411 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.615554 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.615684 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.615808 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.616056 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.616260 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.616401 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.616579 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.616721 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.616867 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.617105 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.617245 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.617378 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.617566 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.617706 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.617838 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.618012 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.618150 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.618281 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.618459 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.618599 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.618785 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.618919 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.619052 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.619330 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.619486 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.619635 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.619773 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.619904 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.620039 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.620164 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.620308 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.620446 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.620629 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.620763 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.620923 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.621057 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.621207 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.621429 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.621649 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.621649 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.622023 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.622221 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.622221 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.622605 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.622799 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.622799 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.623169 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.623367 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.623367 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.623741 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.623934 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.623934 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.624309 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.624558 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.624558 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.624947 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.625143 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.625337 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] token.caching = True {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.625531 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.625531 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.625908 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.626102 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.626102 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.626470 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.628953 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.628953 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.629235 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.629334 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.629334 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.629746 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.629937 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.629937 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] auth.external = None {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.630313 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.630505 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.630690 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.630880 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.630880 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.631245 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.631431 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] auth.password = None {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.631627 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.631803 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] auth.token = None {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.631803 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.632145 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.632330 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.632535 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.632708 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.632708 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.633041 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.633233 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.633419 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.633597 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.633775 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.633954 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.634158 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.634287 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] credential.caching = True {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.634417 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.634500 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.634639 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.634742 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.634860 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.634984 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.635100 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] database.connection = **** {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.635209 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.635335 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.635444 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.635552 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.635660 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.635766 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.635933 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.636049 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.636049 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.636236 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.636358 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.636434 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.636627 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.636717 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.636791 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.636909 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.636985 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.637100 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.637185 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.637185 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.637391 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.637473 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.637616 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.637694 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.637819 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.637897 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.638013 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] federation.caching = True {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.638120 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.638233 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.638350 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.638494 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.638627 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.638761 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.638890 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.638890 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 05 19:48:11.639131 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.639131 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.639131 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.639131 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.639131 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.639131 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.639131 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.639131 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.639131 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] identity.caching = True {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.640140 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.640140 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.640140 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.640140 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.640140 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.640140 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.640140 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.640140 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.640140 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.641317 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.641317 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.641317 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.641317 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.641317 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.641317 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.641317 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.642133 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.642133 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.642133 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.643004 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.643004 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.643239 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.643429 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.643690 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.643959 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.644205 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.644515 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.644771 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.645044 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.645319 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.645589 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.645858 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.646111 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.646346 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.646591 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.646826 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.647071 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.647313 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.647563 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.647806 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.648049 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.648302 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.648654 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.648890 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.649125 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.649377 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.649883 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.650124 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.650363 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.650602 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.650836 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] ldap.user = None {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.651079 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.651341 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.651575 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.651823 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.652073 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.652317 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.652622 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.652880 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.653143 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.653402 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.653665 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.653934 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.654210 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.654493 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.654764 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.655011 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.655299 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.655569 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.655859 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.656136 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.656395 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.656682 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.656955 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.657244 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.657519 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.657765 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.658581 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.658848 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.659125 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.659387 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.659653 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.659905 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.660352 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.660643 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.660912 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.661185 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.661434 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] resource.caching = True {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.661727 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.662005 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.662336 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.662566 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.662828 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.663109 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.663361 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.663627 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.663892 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.664178 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] role.caching = True {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.664451 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] role.driver = sql {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.664768 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.665066 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.665343 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.665631 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.665882 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80363) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 19:48:11.666038 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.666192 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80363) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 19:48:11.666333 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.666472 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80363) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 19:48:11.666615 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.666754 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.666934 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.667201 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.667475 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.667765 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.668018 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.668268 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.668765 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.669031 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.669325 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.669604 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.669879 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.670158 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.670401 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.670664 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.670921 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.671192 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.671451 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.671687 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.671922 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.672201 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.672481 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.672792 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.673052 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.673317 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.673605 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.673847 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] token.caching = True {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.674089 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.674346 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.674615 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.674879 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.675105 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80363) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 19:48:11.675319 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.675524 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.675734 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.675940 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.676145 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.676364 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.676614 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.676817 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.677067 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.677346 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.677590 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.677876 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.678202 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.678486 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] database.connection = **** {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.678759 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.679049 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.679294 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.679576 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.679840 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.680121 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.680364 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.680687 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.680908 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.681193 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.681419 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.681667 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.681918 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.682161 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.682414 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.682649 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.682870 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.683113 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 05 19:48:11.683302 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 05 19:48:11.705650 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80363) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 19:48:11.705938 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80363) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 19:48:11.710411 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80364) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 19:48:11.710752 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80364) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 19:48:11.710921 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80364) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 19:48:11.720191 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80364) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 19:48:11.742985 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80364) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 19:48:11.743328 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80364) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 19:48:11.761896 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80363) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 19:48:11.784690 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80364) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 19:48:11.815055 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80363) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 19:48:11.815370 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80363) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 19:48:11.825140 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80364) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 19:48:11.825510 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80364) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 19:48:11.883799 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80363) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 19:48:11.884095 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80363) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 19:48:11.884264 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80363) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 19:48:11.884430 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80363) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 19:48:11.884632 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80363) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 19:48:11.884794 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80363) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 19:48:11.885286 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80364) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 19:48:11.885523 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80364) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 19:48:11.885751 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80364) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 19:48:11.885971 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80364) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 19:48:11.886192 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80364) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 19:48:11.886414 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80364) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 19:48:11.910121 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80363) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 19:48:11.910411 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80363) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 19:48:11.910580 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80363) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 19:48:11.910761 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80363) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 19:48:11.910924 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80364) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 19:48:11.911087 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80363) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 19:48:11.911247 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80364) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 19:48:11.911375 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80363) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 19:48:11.911506 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80364) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 19:48:11.911754 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80363) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 19:48:11.911909 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80364) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 19:48:11.911909 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80364) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 19:48:11.911909 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80364) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 19:48:11.912336 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80364) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 19:48:11.923843 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80363) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 19:48:11.924170 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80364) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 19:48:11.937197 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80363) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 19:48:11.937536 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80363) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 19:48:11.937773 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80363) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 19:48:11.938100 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80363) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 19:48:11.938251 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80364) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 19:48:11.938395 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80364) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 19:48:11.938531 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80363) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 19:48:11.938668 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80364) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 19:48:11.938826 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80364) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 19:48:11.938960 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80363) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 19:48:11.939094 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80364) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 19:48:11.939714 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80363) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 19:48:11.939846 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80364) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 19:48:11.939988 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80364) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 05 19:48:11.958139 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:48:11.958393 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 05 19:48:11.958660 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:48:11.958857 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 05 19:48:11.959003 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80364) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:48:11.959178 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80364) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 05 19:48:11.959310 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:48:11.959398 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:48:11.959504 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80364) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:48:11.959626 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:48:11.959759 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80364) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 05 19:48:11.959879 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:48:11.959997 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80364) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:48:11.960255 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80364) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:48:11.960478 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80364) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:48:11.960815 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80364) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:48:11.969558 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:48:11.969833 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:48:11.970101 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:48:11.970406 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:48:11.971347 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80364) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:48:11.971653 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80364) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:48:11.971957 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80364) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:48:11.972267 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80364) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:48:11.977502 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80363) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 05 19:48:11.979776 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80364) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 05 19:48:11.982584 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80363) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 05 19:48:11.982966 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:48:11.983324 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:48:11.983681 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:48:11.984040 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:48:11.984396 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:48:11.984782 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:48:11.984930 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80364) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 05 19:48:11.985194 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:48:11.985343 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80364) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:48:11.985560 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80364) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:48:11.985712 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:48:11.985851 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80364) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:48:11.986096 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:48:11.986242 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80364) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:48:11.986377 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80364) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:48:11.986541 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:48:11.986698 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80364) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:48:11.986884 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80364) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:48:11.987160 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80364) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:48:11.987435 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80364) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:48:11.987703 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80364) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:48:12.010835 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:48:12.012353 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80363) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 05 19:48:12.012555 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80364) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:48:12.012705 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:48:12.013975 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80364) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 05 19:48:12.014258 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80364) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:48:12.017303 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80363) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 05 19:48:12.018100 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:48:12.018383 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:48:12.019622 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80364) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 05 19:48:12.019977 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80364) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:48:12.020360 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80364) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:48:12.025346 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80363) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 05 19:48:12.025627 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:48:12.027931 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80364) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 05 19:48:12.028320 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80364) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:48:12.029773 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80363) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 05 19:48:12.030059 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:48:12.030345 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:48:12.030626 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:48:12.030905 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:48:12.031200 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:48:12.031466 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:48:12.031738 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:48:12.034082 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80364) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 05 19:48:12.034436 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80364) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:48:12.034801 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80364) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:48:12.035145 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80364) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:48:12.035501 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80364) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:48:12.035867 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80364) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:48:12.036201 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80364) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:48:12.036589 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80364) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:48:12.047830 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:48:12.048120 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:48:12.052159 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80363) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 05 19:48:12.054056 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80364) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:48:12.054396 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80364) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:48:12.055363 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:48:12.055654 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:48:12.058645 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80364) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 05 19:48:12.060088 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80363) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 05 19:48:12.062203 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80364) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:48:12.062549 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80364) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:48:12.063303 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80363) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 05 19:48:12.067002 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:48:12.067299 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:48:12.067580 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:48:12.067872 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:48:12.068001 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80364) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 05 19:48:12.068157 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:48:12.068473 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:48:12.071882 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80364) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 05 19:48:12.075400 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80364) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:48:12.075729 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80364) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:48:12.076045 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80364) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:48:12.076355 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80364) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:48:12.076686 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80364) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:48:12.077006 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80364) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:48:12.085218 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80363) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 05 19:48:12.085511 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:48:12.085774 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:48:12.086041 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:48:12.092996 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:48:12.094561 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:48:12.094753 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80364) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 05 19:48:12.094913 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:48:12.095097 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80364) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:48:12.095467 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80364) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:48:12.095784 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80364) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:48:12.097708 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80363) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 05 19:48:12.097989 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:48:12.098276 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:48:12.098557 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:48:12.098841 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:48:12.102971 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80364) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:48:12.104575 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80364) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:48:12.104851 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80364) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:48:12.107740 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80364) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 05 19:48:12.108040 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80364) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:48:12.108337 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80364) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:48:12.108647 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80364) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:48:12.108939 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80364) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:48:12.110416 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80363) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 05 19:48:12.110827 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:48:12.111057 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:48:12.111283 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:48:12.111562 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:48:12.111832 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:48:12.112113 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:48:12.120724 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80364) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 05 19:48:12.121008 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80364) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:48:12.121281 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80364) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:48:12.121565 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80364) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:48:12.121846 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80364) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:48:12.122116 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80364) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:48:12.122403 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80364) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:48:12.129373 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80363) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 05 19:48:12.132414 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80363) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 05 19:48:12.135492 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:48:12.138138 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:48:12.138958 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80364) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 05 19:48:12.139770 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80363) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 05 19:48:12.140063 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:48:12.140352 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:48:12.142053 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80364) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 05 19:48:12.145627 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80364) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:48:12.147465 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:48:12.148306 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80364) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:48:12.149125 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80363) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 05 19:48:12.150238 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80364) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 05 19:48:12.150581 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80364) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:48:12.150887 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80364) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:48:12.152341 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:48:12.152652 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:48:12.152931 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:48:12.153199 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:48:12.157622 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80364) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:48:12.159285 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80364) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 05 19:48:12.161404 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80363) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 05 19:48:12.161693 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:48:12.161970 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:48:12.162321 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80364) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:48:12.162647 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80364) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:48:12.162954 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80364) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:48:12.163232 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80364) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:48:12.168919 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80363) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 05 19:48:12.169204 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:48:12.169480 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:48:12.169750 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:48:12.170017 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:48:12.170284 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:48:12.170617 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:48:12.170839 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:48:12.171198 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:48:12.171420 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80364) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 05 19:48:12.171571 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:48:12.171732 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:48:12.171880 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80364) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:48:12.172028 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:48:12.172177 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80364) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:48:12.172325 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:48:12.172473 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:48:12.179183 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80364) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 05 19:48:12.179465 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80364) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:48:12.179745 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80364) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:48:12.180021 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80364) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:48:12.180293 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80364) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:48:12.180593 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80364) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:48:12.180871 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80364) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:48:12.181150 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80364) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:48:12.181550 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80364) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:48:12.181825 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80364) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:48:12.182114 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80364) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:48:12.182379 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80364) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:48:12.182657 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80364) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:48:12.182918 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80364) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 05 19:48:12.207312 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: WSGI app 0 (mountpoint='') ready in 7 seconds on interpreter 0x55bb4a6289c0 pid: 80363 (default app) Aug 05 19:48:12.210842 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3899c3-216e-40cb-a3fe-a32ffa93273d None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:48:12.211135 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3899c3-216e-40cb-a3fe-a32ffa93273d None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:48:12.211411 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3899c3-216e-40cb-a3fe-a32ffa93273d None None] PATH_INFO: `/v3` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:48:12.215795 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1/1] 10.222.0.27 () {50 vars in 791 bytes} [Fri Aug 5 19:48:12 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:48:12.218212 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: WSGI app 0 (mountpoint='') ready in 7 seconds on interpreter 0x55bb4a6289c0 pid: 80364 (default app) Aug 05 19:48:20.629468 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6dd8826-ca11-4760-aa74-97346eb2027f None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:48:20.630255 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6dd8826-ca11-4760-aa74-97346eb2027f None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:48:20.630395 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6dd8826-ca11-4760-aa74-97346eb2027f None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:48:20.632329 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 2/2] 10.222.0.27 () {54 vars in 927 bytes} [Fri Aug 5 19:48:20 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 05 19:48:20.640554 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8425ae73-acf3-4439-8f05-85c37d3a98da None None] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:48:20.641104 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8425ae73-acf3-4439-8f05-85c37d3a98da None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:48:20.641417 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8425ae73-acf3-4439-8f05-85c37d3a98da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:48:20.750245 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: INFO dbcounter [None req-8425ae73-acf3-4439-8f05-85c37d3a98da None None] Registered counter for database keystone Aug 05 19:48:20.771020 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG oslo_db.sqlalchemy.engines [None req-8425ae73-acf3-4439-8f05-85c37d3a98da None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80364) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 05 19:48:20.778055 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writer thread running {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 05 19:48:20.969772 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG passlib.handlers.bcrypt [None req-8425ae73-acf3-4439-8f05-85c37d3a98da None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80364) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Aug 05 19:48:20.970097 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG passlib.handlers.bcrypt [None req-8425ae73-acf3-4439-8f05-85c37d3a98da None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80364) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Aug 05 19:48:21.034113 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.auth.core [None req-8425ae73-acf3-4439-8f05-85c37d3a98da None None] MFA Rules not processed for user `03d1834a64cd4a43829827cb99a72747`. Rule list: `[]` (Enabled: `True`). {{(pid=80364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:48:21.036024 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-8425ae73-acf3-4439-8f05-85c37d3a98da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:48:21.212321 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1/3] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Aug 5 19:48:20 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 575 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Aug 05 19:48:21.410477 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6dd8826-ca11-4760-aa74-97346eb2027f None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:48:21.481541 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: INFO dbcounter [None req-b6dd8826-ca11-4760-aa74-97346eb2027f None None] Registered counter for database keystone Aug 05 19:48:21.505180 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG oslo_db.sqlalchemy.engines [None req-b6dd8826-ca11-4760-aa74-97346eb2027f None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80363) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 05 19:48:21.514679 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writer thread running {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 05 19:48:21.675494 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6dd8826-ca11-4760-aa74-97346eb2027f None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:48:21.680316 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d341f2-1956-4543-b9dc-954a48e8bf87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd25cd7afe64843529ae31496a857a541', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:48:21.681007 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d341f2-1956-4543-b9dc-954a48e8bf87 admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:48:21.681292 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d341f2-1956-4543-b9dc-954a48e8bf87 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:48:21.681569 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d341f2-1956-4543-b9dc-954a48e8bf87 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:48:21.690706 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d341f2-1956-4543-b9dc-954a48e8bf87 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:48:21.743707 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: WARNING keystone.common.rbac_enforcer.enforcer [None req-81d341f2-1956-4543-b9dc-954a48e8bf87 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Aug 05 19:48:21.744045 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d341f2-1956-4543-b9dc-954a48e8bf87 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:48:21.757296 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 3/4] 10.222.0.27 () {58 vars in 1324 bytes} [Fri Aug 5 19:48:21 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 05 19:48:21.768386 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8425ae73-acf3-4439-8f05-85c37d3a98da None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:48:21.793153 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8425ae73-acf3-4439-8f05-85c37d3a98da None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:48:21.799854 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c690cdb2-e6a8-44af-936f-c69d0b45d462 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd25cd7afe64843529ae31496a857a541', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:48:21.801747 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c690cdb2-e6a8-44af-936f-c69d0b45d462 admin admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:48:21.802217 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c690cdb2-e6a8-44af-936f-c69d0b45d462 admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:48:21.802631 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c690cdb2-e6a8-44af-936f-c69d0b45d462 admin admin] PATH_INFO: `/v3/projects/d25cd7afe64843529ae31496a857a541` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:48:21.805181 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c690cdb2-e6a8-44af-936f-c69d0b45d462 admin admin] RBAC: Authorizing `identity:get_project(project_id=d25cd7afe64843529ae31496a857a541)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:48:21.869956 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: WARNING keystone.common.rbac_enforcer.enforcer [None req-c690cdb2-e6a8-44af-936f-c69d0b45d462 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Aug 05 19:48:21.870295 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c690cdb2-e6a8-44af-936f-c69d0b45d462 admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:48:21.875074 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 2/5] 10.222.0.27 () {56 vars in 1207 bytes} [Fri Aug 5 19:48:21 2022] GET /identity/v3/projects/d25cd7afe64843529ae31496a857a541 => generated 344 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 19:48:23.950044 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c966e443-b6fe-4a84-9414-5c7ef3736122 None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:48:23.950612 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c966e443-b6fe-4a84-9414-5c7ef3736122 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:48:23.950612 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c966e443-b6fe-4a84-9414-5c7ef3736122 None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:48:23.954858 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 4/6] 10.222.0.27 () {54 vars in 927 bytes} [Fri Aug 5 19:48:23 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:48:23.961059 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c346b5-a122-461a-b6a3-f29382a7eaae None None] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:48:23.961565 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c346b5-a122-461a-b6a3-f29382a7eaae None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:48:23.961655 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c346b5-a122-461a-b6a3-f29382a7eaae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:48:23.983068 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.auth.core [None req-16c346b5-a122-461a-b6a3-f29382a7eaae None None] MFA Rules not processed for user `03d1834a64cd4a43829827cb99a72747`. Rule list: `[]` (Enabled: `True`). {{(pid=80364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:48:23.985350 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-16c346b5-a122-461a-b6a3-f29382a7eaae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:48:23.994033 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 3/7] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Aug 5 19:48:23 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Aug 05 19:48:24.016753 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c966e443-b6fe-4a84-9414-5c7ef3736122 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:48:24.030051 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c966e443-b6fe-4a84-9414-5c7ef3736122 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:48:24.035068 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1e0538-5341-405c-85cc-3edb5530b84a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd25cd7afe64843529ae31496a857a541', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:48:24.035717 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1e0538-5341-405c-85cc-3edb5530b84a admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:48:24.036001 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1e0538-5341-405c-85cc-3edb5530b84a admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:48:24.036280 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1e0538-5341-405c-85cc-3edb5530b84a admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:48:24.045163 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1e0538-5341-405c-85cc-3edb5530b84a admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:48:24.047078 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1e0538-5341-405c-85cc-3edb5530b84a admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:48:24.060781 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 5/8] 10.222.0.27 () {58 vars in 1324 bytes} [Fri Aug 5 19:48:24 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 05 19:48:24.068051 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c346b5-a122-461a-b6a3-f29382a7eaae None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:48:24.089716 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c346b5-a122-461a-b6a3-f29382a7eaae None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:48:24.096966 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2a3d1e-db1d-4a38-857b-5afba8e4fcf3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd25cd7afe64843529ae31496a857a541', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:48:24.097951 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2a3d1e-db1d-4a38-857b-5afba8e4fcf3 admin admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:48:24.098384 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2a3d1e-db1d-4a38-857b-5afba8e4fcf3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:48:24.098804 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2a3d1e-db1d-4a38-857b-5afba8e4fcf3 admin admin] PATH_INFO: `/v3/users/03d1834a64cd4a43829827cb99a72747` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:48:24.101105 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2a3d1e-db1d-4a38-857b-5afba8e4fcf3 admin admin] RBAC: Authorizing `identity:get_user(user_id=03d1834a64cd4a43829827cb99a72747)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:48:24.104111 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2a3d1e-db1d-4a38-857b-5afba8e4fcf3 admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:48:24.106726 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 4/9] 10.222.0.27 () {56 vars in 1198 bytes} [Fri Aug 5 19:48:24 2022] GET /identity/v3/users/03d1834a64cd4a43829827cb99a72747 => generated 247 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 19:48:26.233602 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0294d9e5-cff9-4504-a454-80698879c09d None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:48:26.234147 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0294d9e5-cff9-4504-a454-80698879c09d None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:48:26.234147 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0294d9e5-cff9-4504-a454-80698879c09d None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:48:26.235526 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 6/10] 10.222.0.27 () {54 vars in 927 bytes} [Fri Aug 5 19:48:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:48:26.241684 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f91273a-39b8-4ef8-b9ef-2e66b8ed5171 None None] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:48:26.242209 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f91273a-39b8-4ef8-b9ef-2e66b8ed5171 None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:48:26.242298 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f91273a-39b8-4ef8-b9ef-2e66b8ed5171 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:48:26.261969 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.auth.core [None req-6f91273a-39b8-4ef8-b9ef-2e66b8ed5171 None None] MFA Rules not processed for user `03d1834a64cd4a43829827cb99a72747`. Rule list: `[]` (Enabled: `True`). {{(pid=80364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:48:26.263173 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-6f91273a-39b8-4ef8-b9ef-2e66b8ed5171 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:48:26.275709 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea02d1c-4b7a-472a-af8a-950fae930ce2 None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:48:26.276076 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea02d1c-4b7a-472a-af8a-950fae930ce2 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:48:26.276423 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea02d1c-4b7a-472a-af8a-950fae930ce2 None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:48:26.278040 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 7/11] 10.222.0.27 () {54 vars in 927 bytes} [Fri Aug 5 19:48:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:48:26.284439 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9087b4-3d9b-4285-9706-fa750787bf67 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:48:26.285154 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9087b4-3d9b-4285-9706-fa750787bf67 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:48:26.285154 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9087b4-3d9b-4285-9706-fa750787bf67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:48:26.328241 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG passlib.handlers.bcrypt [None req-8f9087b4-3d9b-4285-9706-fa750787bf67 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80363) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Aug 05 19:48:26.328700 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG passlib.handlers.bcrypt [None req-8f9087b4-3d9b-4285-9706-fa750787bf67 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80363) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Aug 05 19:48:26.348922 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 5/12] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Aug 5 19:48:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 05 19:48:26.372288 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f91273a-39b8-4ef8-b9ef-2e66b8ed5171 None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:48:26.375302 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-8f9087b4-3d9b-4285-9706-fa750787bf67 None None] MFA Rules not processed for user `03d1834a64cd4a43829827cb99a72747`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:48:26.376966 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-8f9087b4-3d9b-4285-9706-fa750787bf67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:48:26.398637 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:48:26.481655 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 8/13] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Aug 5 19:48:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 05 19:48:26.484981 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f91273a-39b8-4ef8-b9ef-2e66b8ed5171 None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:48:26.503919 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9087b4-3d9b-4285-9706-fa750787bf67 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:48:26.521060 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9436db7-8e69-4be3-9935-f89cfb65afbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:48:26.521713 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9436db7-8e69-4be3-9935-f89cfb65afbe None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:48:26.521990 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9436db7-8e69-4be3-9935-f89cfb65afbe None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:48:26.522264 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9436db7-8e69-4be3-9935-f89cfb65afbe None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:48:26.535091 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9436db7-8e69-4be3-9935-f89cfb65afbe None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 19:48:26.543970 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application [None req-f9436db7-8e69-4be3-9935-f89cfb65afbe None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 19:48:26.543970 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:48:26.543970 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:48:26.543970 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:48:26.543970 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:48:26.543970 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:48:26.543970 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:48:26.543970 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:48:26.543970 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:48:26.543970 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:48:26.543970 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:48:26.543970 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:48:26.543970 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 19:48:26.543970 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 19:48:26.543970 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 19:48:26.543970 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 19:48:26.545031 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 19:48:26.545031 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 19:48:26.545031 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 19:48:26.545031 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 19:48:26.545031 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:48:26.545031 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:48:26.545031 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:48:26.545031 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:48:26.545031 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:48:26.545031 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:48:26.545031 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:48:26.545031 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application with Lock( Aug 05 19:48:26.545031 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:48:26.545031 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:48:26.545031 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:48:26.545031 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:48:26.545883 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:48:26.545883 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:48:26.545883 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:48:26.545883 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:48:26.545883 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 19:48:26.545883 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 19:48:26.545883 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 19:48:26.545883 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 19:48:26.545883 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 19:48:26.545883 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 19:48:26.545883 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 19:48:26.545883 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Aug 05 19:48:26.545883 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 6/14] 10.222.0.27 () {56 vars in 1096 bytes} [Fri Aug 5 19:48:26 2022] GET /identity/v3/roles/admin => generated 83 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 05 19:48:26.551268 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9436db7-8e69-4be3-9935-f89cfb65afbe None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:48:26.610196 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9087b4-3d9b-4285-9706-fa750787bf67 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:48:26.612679 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:48:26.643716 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:48:26.664712 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9436db7-8e69-4be3-9935-f89cfb65afbe None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:48:26.672804 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ba79e5-2a05-4c1b-9968-7d1168af5cd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:48:26.674402 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ba79e5-2a05-4c1b-9968-7d1168af5cd5 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:48:26.674812 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ba79e5-2a05-4c1b-9968-7d1168af5cd5 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:48:26.675224 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ba79e5-2a05-4c1b-9968-7d1168af5cd5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:48:26.701090 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5052a40-0950-46cb-bcff-cd1aa77dc6cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:48:26.701727 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5052a40-0950-46cb-bcff-cd1aa77dc6cd None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:48:26.702001 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5052a40-0950-46cb-bcff-cd1aa77dc6cd None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:48:26.702273 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5052a40-0950-46cb-bcff-cd1aa77dc6cd None admin] PATH_INFO: `/v3/roles` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:48:26.703069 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5052a40-0950-46cb-bcff-cd1aa77dc6cd None admin] RBAC: Adding query filter params (name=admin) {{(pid=80364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:48:26.703375 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5052a40-0950-46cb-bcff-cd1aa77dc6cd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:48:26.705289 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5052a40-0950-46cb-bcff-cd1aa77dc6cd None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:48:26.717965 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 7/15] 10.222.0.27 () {56 vars in 1099 bytes} [Fri Aug 5 19:48:26 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 19:48:26.724257 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5052a40-0950-46cb-bcff-cd1aa77dc6cd None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:48:26.750515 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ba79e5-2a05-4c1b-9968-7d1168af5cd5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:48:26.752003 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ba79e5-2a05-4c1b-9968-7d1168af5cd5 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:48:26.868610 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 9/16] 10.222.0.27 () {58 vars in 1282 bytes} [Fri Aug 5 19:48:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 05 19:48:26.874864 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ba79e5-2a05-4c1b-9968-7d1168af5cd5 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:48:26.878211 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:48:26.944864 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5052a40-0950-46cb-bcff-cd1aa77dc6cd None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:48:26.980234 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ba79e5-2a05-4c1b-9968-7d1168af5cd5 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:48:26.981180 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dfc70b2-d752-472c-bbb3-3b6c3667ecb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:48:26.981841 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfc70b2-d752-472c-bbb3-3b6c3667ecb2 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:48:26.982129 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfc70b2-d752-472c-bbb3-3b6c3667ecb2 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:48:26.982473 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfc70b2-d752-472c-bbb3-3b6c3667ecb2 None admin] PATH_INFO: `/v3/users/03d1834a64cd4a43829827cb99a72747` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:48:26.984048 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dfc70b2-d752-472c-bbb3-3b6c3667ecb2 None admin] RBAC: Authorizing `identity:get_user(user_id=03d1834a64cd4a43829827cb99a72747)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:48:26.985729 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dfc70b2-d752-472c-bbb3-3b6c3667ecb2 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:48:26.987355 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 8/17] 10.222.0.27 () {56 vars in 1177 bytes} [Fri Aug 5 19:48:26 2022] GET /identity/v3/users/03d1834a64cd4a43829827cb99a72747 => generated 247 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 19:48:26.993436 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dfc70b2-d752-472c-bbb3-3b6c3667ecb2 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:48:27.010399 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:48:27.019552 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8144ca73-f87a-4c41-bdbf-5078a2df58d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:48:27.020423 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8144ca73-f87a-4c41-bdbf-5078a2df58d2 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:48:27.020738 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8144ca73-f87a-4c41-bdbf-5078a2df58d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:48:27.021016 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8144ca73-f87a-4c41-bdbf-5078a2df58d2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:48:27.040651 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8144ca73-f87a-4c41-bdbf-5078a2df58d2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:48:27.042187 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8144ca73-f87a-4c41-bdbf-5078a2df58d2 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:48:27.061329 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-8144ca73-f87a-4c41-bdbf-5078a2df58d2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:48:27.061329 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:48:27.061329 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:48:27.061329 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:48:27.061329 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:48:27.061329 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:48:27.061329 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:48:27.061329 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:48:27.061329 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:48:27.061329 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:48:27.061329 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:48:27.061329 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:48:27.061329 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 19:48:27.061329 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 19:48:27.061329 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 19:48:27.062314 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 19:48:27.062314 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:48:27.062314 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:48:27.062314 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:48:27.062314 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:48:27.062314 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:48:27.062314 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:48:27.062314 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:48:27.062314 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Aug 05 19:48:27.062314 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:48:27.062314 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:48:27.062314 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:48:27.062314 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:48:27.062314 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:48:27.062314 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:48:27.062314 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:48:27.062314 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:48:27.063100 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 19:48:27.063100 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 19:48:27.063100 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:48:27.063100 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Aug 05 19:48:27.063100 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 10/18] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Aug 5 19:48:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 05 19:48:27.068628 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8144ca73-f87a-4c41-bdbf-5078a2df58d2 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:48:27.108886 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dfc70b2-d752-472c-bbb3-3b6c3667ecb2 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:48:27.132390 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:48:27.146245 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e479b0-afcf-4d7f-b705-dae88c916cd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:48:27.146899 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e479b0-afcf-4d7f-b705-dae88c916cd2 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:48:27.147178 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e479b0-afcf-4d7f-b705-dae88c916cd2 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:48:27.147456 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e479b0-afcf-4d7f-b705-dae88c916cd2 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:48:27.148669 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e479b0-afcf-4d7f-b705-dae88c916cd2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:48:27.150409 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e479b0-afcf-4d7f-b705-dae88c916cd2 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:48:27.152009 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 9/19] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Aug 5 19:48:26 2022] GET /identity/v3/domains/default => generated 199 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 19:48:27.158310 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e479b0-afcf-4d7f-b705-dae88c916cd2 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:48:27.181374 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8144ca73-f87a-4c41-bdbf-5078a2df58d2 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:48:27.218389 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e0b416-3fca-40c4-b023-4128e568a938 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:48:27.219021 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e0b416-3fca-40c4-b023-4128e568a938 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:48:27.219308 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e0b416-3fca-40c4-b023-4128e568a938 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:48:27.219581 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e0b416-3fca-40c4-b023-4128e568a938 None admin] PATH_INFO: `/v3/domains` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:48:27.220323 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e0b416-3fca-40c4-b023-4128e568a938 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:48:27.221439 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e0b416-3fca-40c4-b023-4128e568a938 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:48:27.222104 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e0b416-3fca-40c4-b023-4128e568a938 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:48:27.241413 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 11/20] 10.222.0.27 () {56 vars in 1109 bytes} [Fri Aug 5 19:48:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 19:48:27.242519 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:48:27.265528 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e479b0-afcf-4d7f-b705-dae88c916cd2 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:48:27.303993 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f9641c-fdd1-4dbb-b48a-57be79a5477b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:48:27.304670 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f9641c-fdd1-4dbb-b48a-57be79a5477b None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:48:27.304955 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f9641c-fdd1-4dbb-b48a-57be79a5477b None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:48:27.305233 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f9641c-fdd1-4dbb-b48a-57be79a5477b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:48:27.306043 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f9641c-fdd1-4dbb-b48a-57be79a5477b None admin] RBAC: Adding query filter params (role.id=c173688a960b46e5be65f0ccd6fbd734, scope.domain.id=default, user.id=03d1834a64cd4a43829827cb99a72747) {{(pid=80364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:48:27.306368 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f9641c-fdd1-4dbb-b48a-57be79a5477b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:48:27.307857 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f9641c-fdd1-4dbb-b48a-57be79a5477b None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:48:27.316532 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 10/21] 10.222.0.27 () {56 vars in 1322 bytes} [Fri Aug 5 19:48:27 2022] GET /identity/v3/role_assignments?user.id=03d1834a64cd4a43829827cb99a72747&scope.domain.id=default&role.id=c173688a960b46e5be65f0ccd6fbd734 => generated 233 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 19:48:29.537773 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507d4a6e-deac-4bdc-a7a8-139ba4806ffd None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:48:29.538297 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507d4a6e-deac-4bdc-a7a8-139ba4806ffd None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:48:29.538297 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507d4a6e-deac-4bdc-a7a8-139ba4806ffd None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:48:29.539680 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 12/22] 10.222.0.27 () {54 vars in 927 bytes} [Fri Aug 5 19:48:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:48:29.545591 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afb4554-1ae5-4b47-81b1-6a0c0f646bab None None] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:48:29.546089 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afb4554-1ae5-4b47-81b1-6a0c0f646bab None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:48:29.546163 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afb4554-1ae5-4b47-81b1-6a0c0f646bab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:48:29.569615 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.auth.core [None req-0afb4554-1ae5-4b47-81b1-6a0c0f646bab None None] MFA Rules not processed for user `03d1834a64cd4a43829827cb99a72747`. Rule list: `[]` (Enabled: `True`). {{(pid=80364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:48:29.570827 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-0afb4554-1ae5-4b47-81b1-6a0c0f646bab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:48:29.666876 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 11/23] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Aug 5 19:48:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 05 19:48:29.708927 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507d4a6e-deac-4bdc-a7a8-139ba4806ffd None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:48:29.857558 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507d4a6e-deac-4bdc-a7a8-139ba4806ffd None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:48:29.875244 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:48:29.903177 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389b2989-f785-4b8e-98b8-7e8fae97eca4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:48:29.903826 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389b2989-f785-4b8e-98b8-7e8fae97eca4 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:48:29.904105 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389b2989-f785-4b8e-98b8-7e8fae97eca4 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:48:29.904377 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389b2989-f785-4b8e-98b8-7e8fae97eca4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:48:29.920876 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-389b2989-f785-4b8e-98b8-7e8fae97eca4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 19:48:29.927378 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-389b2989-f785-4b8e-98b8-7e8fae97eca4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 19:48:29.927378 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:48:29.927378 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:48:29.927378 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:48:29.927378 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:48:29.927378 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:48:29.927378 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:48:29.927378 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:48:29.927378 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:48:29.927378 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:48:29.927378 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:48:29.927378 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:48:29.927378 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 19:48:29.927378 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 19:48:29.927378 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 19:48:29.927378 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 19:48:29.928226 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 19:48:29.928226 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 19:48:29.928226 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 19:48:29.928226 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 19:48:29.928226 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:48:29.928226 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:48:29.928226 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:48:29.928226 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:48:29.928226 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:48:29.928226 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:48:29.928226 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:48:29.928226 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Aug 05 19:48:29.928226 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:48:29.928226 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:48:29.928226 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:48:29.928226 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:48:29.929341 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:48:29.929341 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:48:29.929341 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:48:29.929341 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:48:29.929341 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 19:48:29.929341 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 19:48:29.929341 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 19:48:29.929341 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 19:48:29.929341 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 19:48:29.929341 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 19:48:29.929341 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 19:48:29.929341 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Aug 05 19:48:29.929341 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 13/24] 10.222.0.27 () {56 vars in 1096 bytes} [Fri Aug 5 19:48:29 2022] GET /identity/v3/roles/admin => generated 83 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:48:29.935718 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0afb4554-1ae5-4b47-81b1-6a0c0f646bab None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:48:30.051373 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:48:30.087303 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0afb4554-1ae5-4b47-81b1-6a0c0f646bab None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:48:30.134734 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2edb03f0-3b49-44d2-bb25-556f82bfaf22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:48:30.135361 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edb03f0-3b49-44d2-bb25-556f82bfaf22 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:48:30.135646 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edb03f0-3b49-44d2-bb25-556f82bfaf22 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:48:30.135919 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edb03f0-3b49-44d2-bb25-556f82bfaf22 None admin] PATH_INFO: `/v3/roles` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:48:30.136759 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2edb03f0-3b49-44d2-bb25-556f82bfaf22 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:48:30.137068 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2edb03f0-3b49-44d2-bb25-556f82bfaf22 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:48:30.138469 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2edb03f0-3b49-44d2-bb25-556f82bfaf22 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:48:30.150377 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 12/25] 10.222.0.27 () {56 vars in 1099 bytes} [Fri Aug 5 19:48:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:48:30.156124 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389b2989-f785-4b8e-98b8-7e8fae97eca4 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:48:30.240067 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad8d3ef-1a37-4d24-99a3-fd722191fcf4 None None] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:48:30.240317 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad8d3ef-1a37-4d24-99a3-fd722191fcf4 None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:48:30.240591 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad8d3ef-1a37-4d24-99a3-fd722191fcf4 None None] PATH_INFO: `/` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:48:30.241967 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 13/26] 10.222.0.27 () {54 vars in 927 bytes} [Fri Aug 5 19:48:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 05 19:48:30.247642 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe0691e-7605-4d7c-876e-2545c9ce6f86 None None] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:48:30.247931 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe0691e-7605-4d7c-876e-2545c9ce6f86 None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:48:30.248213 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe0691e-7605-4d7c-876e-2545c9ce6f86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:48:30.296695 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.auth.core [None req-cbe0691e-7605-4d7c-876e-2545c9ce6f86 None None] MFA Rules not processed for user `03d1834a64cd4a43829827cb99a72747`. Rule list: `[]` (Enabled: `True`). {{(pid=80364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:48:30.296695 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-cbe0691e-7605-4d7c-876e-2545c9ce6f86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:48:30.301826 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389b2989-f785-4b8e-98b8-7e8fae97eca4 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:48:30.345057 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aaaa1a4-a307-4532-9801-e1c67962f533 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:48:30.346007 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aaaa1a4-a307-4532-9801-e1c67962f533 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:48:30.346459 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aaaa1a4-a307-4532-9801-e1c67962f533 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:48:30.346901 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aaaa1a4-a307-4532-9801-e1c67962f533 None admin] PATH_INFO: `/v3/users/03d1834a64cd4a43829827cb99a72747` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:48:30.348690 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aaaa1a4-a307-4532-9801-e1c67962f533 None admin] RBAC: Authorizing `identity:get_user(user_id=03d1834a64cd4a43829827cb99a72747)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:48:30.350349 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aaaa1a4-a307-4532-9801-e1c67962f533 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:48:30.352427 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 14/27] 10.222.0.27 () {56 vars in 1177 bytes} [Fri Aug 5 19:48:30 2022] GET /identity/v3/users/03d1834a64cd4a43829827cb99a72747 => generated 247 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:48:30.354830 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b4e040-5426-47d0-9f5b-9366366833d3 None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:48:30.355309 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b4e040-5426-47d0-9f5b-9366366833d3 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:48:30.355749 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b4e040-5426-47d0-9f5b-9366366833d3 None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:48:30.357934 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 15/28] 10.222.0.27 () {54 vars in 927 bytes} [Fri Aug 5 19:48:30 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:48:30.358971 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b4e040-5426-47d0-9f5b-9366366833d3 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:48:30.479179 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 14/29] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Aug 5 19:48:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 05 19:48:30.480566 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f56432-8cea-4f5d-b4c5-c84fe8bb863d None None] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:48:30.481039 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f56432-8cea-4f5d-b4c5-c84fe8bb863d None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:48:30.481455 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f56432-8cea-4f5d-b4c5-c84fe8bb863d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:48:30.521058 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.auth.core [None req-f7f56432-8cea-4f5d-b4c5-c84fe8bb863d None None] MFA Rules not processed for user `03d1834a64cd4a43829827cb99a72747`. Rule list: `[]` (Enabled: `True`). {{(pid=80364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:48:30.522676 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-f7f56432-8cea-4f5d-b4c5-c84fe8bb863d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:48:30.537146 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:48:30.598857 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b4e040-5426-47d0-9f5b-9366366833d3 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:48:30.615769 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:48:30.630176 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 15/30] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Aug 5 19:48:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 19:48:30.631938 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f56432-8cea-4f5d-b4c5-c84fe8bb863d None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:48:30.657307 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d124b00-074d-4743-aa99-8dbbd85ebd8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:48:30.658532 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d124b00-074d-4743-aa99-8dbbd85ebd8c None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:48:30.658816 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d124b00-074d-4743-aa99-8dbbd85ebd8c None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:48:30.659094 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d124b00-074d-4743-aa99-8dbbd85ebd8c None admin] PATH_INFO: `/v3/domains/default` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:48:30.660617 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d124b00-074d-4743-aa99-8dbbd85ebd8c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:48:30.662675 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d124b00-074d-4743-aa99-8dbbd85ebd8c None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:48:30.664680 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 16/31] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Aug 5 19:48:30 2022] GET /identity/v3/domains/default => generated 199 bytes in 306 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:48:30.666004 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a87b7b1-c844-4e6c-8feb-b9fe556a745a None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:48:30.666293 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a87b7b1-c844-4e6c-8feb-b9fe556a745a None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:48:30.666571 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a87b7b1-c844-4e6c-8feb-b9fe556a745a None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:48:30.667804 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 17/32] 10.222.0.27 () {54 vars in 927 bytes} [Fri Aug 5 19:48:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:48:30.668522 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a87b7b1-c844-4e6c-8feb-b9fe556a745a None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:48:30.781204 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f56432-8cea-4f5d-b4c5-c84fe8bb863d None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:48:30.793423 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a87b7b1-c844-4e6c-8feb-b9fe556a745a None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:48:30.808004 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:48:30.831024 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80bf881-b702-44a9-af3e-659c9b7ffa4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:48:30.832131 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80bf881-b702-44a9-af3e-659c9b7ffa4f None admin] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:48:30.832668 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80bf881-b702-44a9-af3e-659c9b7ffa4f None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:48:30.833151 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80bf881-b702-44a9-af3e-659c9b7ffa4f None admin] PATH_INFO: `/v3/roles` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:48:30.833322 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70c28e1-0884-4905-8005-66661434389a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:48:30.833949 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70c28e1-0884-4905-8005-66661434389a None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:48:30.834235 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70c28e1-0884-4905-8005-66661434389a None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:48:30.834514 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70c28e1-0884-4905-8005-66661434389a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:48:30.834848 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f80bf881-b702-44a9-af3e-659c9b7ffa4f None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:48:30.836664 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f80bf881-b702-44a9-af3e-659c9b7ffa4f None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:48:30.852009 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c70c28e1-0884-4905-8005-66661434389a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:48:30.854017 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c70c28e1-0884-4905-8005-66661434389a None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:48:30.870293 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-c70c28e1-0884-4905-8005-66661434389a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:48:30.870293 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:48:30.870293 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:48:30.870293 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:48:30.870293 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:48:30.870293 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:48:30.870293 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:48:30.870293 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:48:30.870293 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:48:30.870293 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:48:30.870293 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:48:30.870293 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:48:30.870293 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 19:48:30.870293 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 19:48:30.870293 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 19:48:30.871147 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 19:48:30.871147 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:48:30.871147 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:48:30.871147 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:48:30.871147 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:48:30.871147 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:48:30.871147 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:48:30.871147 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:48:30.871147 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Aug 05 19:48:30.871147 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:48:30.871147 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:48:30.871147 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:48:30.871147 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:48:30.871147 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:48:30.871147 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:48:30.871147 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:48:30.871147 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:48:30.871929 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 19:48:30.871929 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 19:48:30.871929 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:48:30.871929 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Aug 05 19:48:30.871929 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 18/33] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Aug 5 19:48:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:48:30.872666 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70c28e1-0884-4905-8005-66661434389a None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:48:30.976591 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 16/34] 10.222.0.27 () {60 vars in 1131 bytes} [Fri Aug 5 19:48:30 2022] POST /identity/v3/roles => generated 219 bytes in 338 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 19:48:30.976591 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bb0fe3-b761-4024-8e25-a740eca3c3e3 None None] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:48:30.976591 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bb0fe3-b761-4024-8e25-a740eca3c3e3 None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:48:30.976591 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bb0fe3-b761-4024-8e25-a740eca3c3e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:48:30.991369 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.auth.core [None req-88bb0fe3-b761-4024-8e25-a740eca3c3e3 None None] MFA Rules not processed for user `03d1834a64cd4a43829827cb99a72747`. Rule list: `[]` (Enabled: `True`). {{(pid=80364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:48:30.992454 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70c28e1-0884-4905-8005-66661434389a None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:48:30.992997 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-88bb0fe3-b761-4024-8e25-a740eca3c3e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:48:31.039875 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d05cdf3-10cb-4bf2-8586-6530d45c2ad4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:48:31.040704 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d05cdf3-10cb-4bf2-8586-6530d45c2ad4 None admin] REQUEST_METHOD: `PUT` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:48:31.041100 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d05cdf3-10cb-4bf2-8586-6530d45c2ad4 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:48:31.041449 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d05cdf3-10cb-4bf2-8586-6530d45c2ad4 None admin] PATH_INFO: `/v3/domains/default/users/03d1834a64cd4a43829827cb99a72747/roles/c173688a960b46e5be65f0ccd6fbd734` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:48:31.044573 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d05cdf3-10cb-4bf2-8586-6530d45c2ad4 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=03d1834a64cd4a43829827cb99a72747, role_id=c173688a960b46e5be65f0ccd6fbd734)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:48:31.046182 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d05cdf3-10cb-4bf2-8586-6530d45c2ad4 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:48:31.062077 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:48:31.087009 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 17/35] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Aug 5 19:48:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 19:48:31.094195 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78402012-8ac1-460d-893a-6552f7b661d0 None None] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:48:31.094498 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78402012-8ac1-460d-893a-6552f7b661d0 None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:48:31.094773 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78402012-8ac1-460d-893a-6552f7b661d0 None None] PATH_INFO: `/` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:48:31.096926 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 18/36] 10.222.0.27 () {54 vars in 927 bytes} [Fri Aug 5 19:48:31 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:48:31.097673 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78402012-8ac1-460d-893a-6552f7b661d0 None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:48:31.185458 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 19/37] 10.222.0.27 () {58 vars in 1361 bytes} [Fri Aug 5 19:48:30 2022] PUT /identity/v3/domains/default/users/03d1834a64cd4a43829827cb99a72747/roles/c173688a960b46e5be65f0ccd6fbd734 => generated 0 bytes in 313 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 19:48:31.187006 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778b4391-7b83-4333-9db6-9332c1378116 None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:48:31.187308 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778b4391-7b83-4333-9db6-9332c1378116 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:48:31.187588 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778b4391-7b83-4333-9db6-9332c1378116 None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:48:31.192419 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 20/38] 10.222.0.27 () {54 vars in 927 bytes} [Fri Aug 5 19:48:31 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:48:31.196577 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3227dffa-7d13-4656-9002-87d881403035 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:48:31.196577 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3227dffa-7d13-4656-9002-87d881403035 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:48:31.196577 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3227dffa-7d13-4656-9002-87d881403035 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:48:31.217328 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-3227dffa-7d13-4656-9002-87d881403035 None None] MFA Rules not processed for user `03d1834a64cd4a43829827cb99a72747`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:48:31.219515 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-3227dffa-7d13-4656-9002-87d881403035 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:48:31.233370 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78402012-8ac1-460d-893a-6552f7b661d0 None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:48:31.240856 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:48:31.276374 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197e4df5-5f5b-4a7a-87df-3271dc1057b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:48:31.277044 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197e4df5-5f5b-4a7a-87df-3271dc1057b7 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:48:31.277331 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197e4df5-5f5b-4a7a-87df-3271dc1057b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:48:31.277611 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197e4df5-5f5b-4a7a-87df-3271dc1057b7 None admin] PATH_INFO: `/v3/domains` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:48:31.278333 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-197e4df5-5f5b-4a7a-87df-3271dc1057b7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:48:31.278640 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-197e4df5-5f5b-4a7a-87df-3271dc1057b7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:48:31.280194 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-197e4df5-5f5b-4a7a-87df-3271dc1057b7 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:48:31.300216 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 19/39] 10.222.0.27 () {56 vars in 1109 bytes} [Fri Aug 5 19:48:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:48:31.301430 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197e4df5-5f5b-4a7a-87df-3271dc1057b7 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:48:31.304654 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 21/40] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Aug 5 19:48:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 19:48:31.306314 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca62f3d6-1f8a-4cea-a759-4f71f2cc2ab0 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:48:31.307192 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca62f3d6-1f8a-4cea-a759-4f71f2cc2ab0 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:48:31.309025 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca62f3d6-1f8a-4cea-a759-4f71f2cc2ab0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:48:31.334806 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-ca62f3d6-1f8a-4cea-a759-4f71f2cc2ab0 None None] MFA Rules not processed for user `03d1834a64cd4a43829827cb99a72747`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:48:31.336808 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-ca62f3d6-1f8a-4cea-a759-4f71f2cc2ab0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:48:31.365676 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:48:31.425118 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 22/41] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Aug 5 19:48:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 19:48:31.426179 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca62f3d6-1f8a-4cea-a759-4f71f2cc2ab0 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:48:31.431529 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197e4df5-5f5b-4a7a-87df-3271dc1057b7 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:48:31.470434 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67bf9804-4afa-41b2-bb77-43e903b0f94d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:48:31.471064 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67bf9804-4afa-41b2-bb77-43e903b0f94d None admin] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:48:31.471351 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67bf9804-4afa-41b2-bb77-43e903b0f94d None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:48:31.471629 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67bf9804-4afa-41b2-bb77-43e903b0f94d None admin] PATH_INFO: `/v3/roles` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:48:31.472347 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67bf9804-4afa-41b2-bb77-43e903b0f94d None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:48:31.473799 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67bf9804-4afa-41b2-bb77-43e903b0f94d None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:48:31.489613 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 20/42] 10.222.0.27 () {60 vars in 1131 bytes} [Fri Aug 5 19:48:31 2022] POST /identity/v3/roles => generated 225 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 19:48:31.490270 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67bf9804-4afa-41b2-bb77-43e903b0f94d None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:48:31.553107 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:48:31.572873 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca62f3d6-1f8a-4cea-a759-4f71f2cc2ab0 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:48:31.615961 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6d12fb-4165-45c3-82ff-2b579e278840 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:48:31.616710 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6d12fb-4165-45c3-82ff-2b579e278840 None admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:48:31.616816 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6d12fb-4165-45c3-82ff-2b579e278840 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:48:31.617161 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6d12fb-4165-45c3-82ff-2b579e278840 None admin] PATH_INFO: `/v3/projects` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:48:31.617955 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d6d12fb-4165-45c3-82ff-2b579e278840 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:48:31.619672 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d6d12fb-4165-45c3-82ff-2b579e278840 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:48:31.638842 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67bf9804-4afa-41b2-bb77-43e903b0f94d None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:48:31.649676 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:48:31.663368 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 23/43] 10.222.0.27 () {60 vars in 1141 bytes} [Fri Aug 5 19:48:31 2022] POST /identity/v3/projects => generated 301 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 19:48:31.664282 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6d12fb-4165-45c3-82ff-2b579e278840 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:48:31.688516 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7d5cdc-722f-4843-b7b3-fff0c47bfc80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:48:31.688516 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7d5cdc-722f-4843-b7b3-fff0c47bfc80 None admin] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:48:31.688516 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7d5cdc-722f-4843-b7b3-fff0c47bfc80 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:48:31.688516 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7d5cdc-722f-4843-b7b3-fff0c47bfc80 None admin] PATH_INFO: `/v3/roles` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:48:31.688961 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7d5cdc-722f-4843-b7b3-fff0c47bfc80 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:48:31.688961 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7d5cdc-722f-4843-b7b3-fff0c47bfc80 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:48:31.706243 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 21/44] 10.222.0.27 () {60 vars in 1131 bytes} [Fri Aug 5 19:48:31 2022] POST /identity/v3/roles => generated 223 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 19:48:31.818225 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6d12fb-4165-45c3-82ff-2b579e278840 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:48:31.863616 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4a48d1-1570-4f38-81aa-27897977094b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:48:31.864296 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4a48d1-1570-4f38-81aa-27897977094b None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:48:31.864616 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4a48d1-1570-4f38-81aa-27897977094b None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:48:31.864915 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4a48d1-1570-4f38-81aa-27897977094b None admin] PATH_INFO: `/v3/domains/default` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:48:31.866144 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d4a48d1-1570-4f38-81aa-27897977094b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:48:31.867658 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d4a48d1-1570-4f38-81aa-27897977094b None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:48:31.869542 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 24/45] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Aug 5 19:48:31 2022] GET /identity/v3/domains/default => generated 199 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:48:31.875554 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7d5cdc-722f-4843-b7b3-fff0c47bfc80 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:48:32.007959 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7d5cdc-722f-4843-b7b3-fff0c47bfc80 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:48:32.040691 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5882384-e453-4afc-a075-011aa34c1515 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:48:32.041322 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5882384-e453-4afc-a075-011aa34c1515 None admin] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:48:32.041609 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5882384-e453-4afc-a075-011aa34c1515 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:48:32.041890 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5882384-e453-4afc-a075-011aa34c1515 None admin] PATH_INFO: `/v3/projects` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:48:32.042583 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5882384-e453-4afc-a075-011aa34c1515 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:48:32.044070 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5882384-e453-4afc-a075-011aa34c1515 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:48:32.076223 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 22/46] 10.222.0.27 () {60 vars in 1141 bytes} [Fri Aug 5 19:48:31 2022] POST /identity/v3/projects => generated 312 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 19:48:33.275400 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1fc239-b117-4ef9-98f8-3586e18441bf None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:48:33.275916 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1fc239-b117-4ef9-98f8-3586e18441bf None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:48:33.276007 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1fc239-b117-4ef9-98f8-3586e18441bf None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:48:33.278017 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 25/47] 10.222.0.27 () {54 vars in 927 bytes} [Fri Aug 5 19:48:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:48:33.283249 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486f5e9a-1bb6-42fa-8140-ecdc3ddce6d7 None None] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:48:33.283660 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486f5e9a-1bb6-42fa-8140-ecdc3ddce6d7 None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:48:33.283820 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486f5e9a-1bb6-42fa-8140-ecdc3ddce6d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:48:33.302519 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.auth.core [None req-486f5e9a-1bb6-42fa-8140-ecdc3ddce6d7 None None] MFA Rules not processed for user `03d1834a64cd4a43829827cb99a72747`. Rule list: `[]` (Enabled: `True`). {{(pid=80364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:48:33.303708 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-486f5e9a-1bb6-42fa-8140-ecdc3ddce6d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:48:33.317409 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:48:33.375143 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 23/48] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Aug 5 19:48:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 19:48:33.398341 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1fc239-b117-4ef9-98f8-3586e18441bf None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:48:33.402461 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:48:33.501941 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1fc239-b117-4ef9-98f8-3586e18441bf None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:48:33.534574 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed16eb3-aa3d-4999-b8f2-35ece39fa8bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:48:33.535147 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed16eb3-aa3d-4999-b8f2-35ece39fa8bb None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:48:33.535434 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed16eb3-aa3d-4999-b8f2-35ece39fa8bb None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:48:33.535712 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed16eb3-aa3d-4999-b8f2-35ece39fa8bb None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:48:33.546070 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed16eb3-aa3d-4999-b8f2-35ece39fa8bb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 19:48:33.551971 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-1ed16eb3-aa3d-4999-b8f2-35ece39fa8bb None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 19:48:33.551971 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:48:33.551971 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:48:33.551971 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:48:33.551971 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:48:33.551971 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:48:33.551971 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:48:33.551971 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:48:33.551971 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:48:33.551971 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:48:33.551971 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:48:33.551971 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:48:33.551971 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 19:48:33.551971 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 19:48:33.551971 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 19:48:33.551971 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 19:48:33.552849 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 19:48:33.552849 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 19:48:33.552849 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 19:48:33.552849 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 19:48:33.552849 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:48:33.552849 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:48:33.552849 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:48:33.552849 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:48:33.552849 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:48:33.552849 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:48:33.552849 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:48:33.552849 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Aug 05 19:48:33.552849 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:48:33.552849 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:48:33.552849 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:48:33.552849 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:48:33.553873 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:48:33.553873 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:48:33.553873 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:48:33.553873 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:48:33.553873 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 19:48:33.553873 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 19:48:33.553873 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 19:48:33.553873 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 19:48:33.553873 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 19:48:33.553873 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 19:48:33.553873 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 19:48:33.553873 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Aug 05 19:48:33.553873 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 26/49] 10.222.0.27 () {56 vars in 1096 bytes} [Fri Aug 5 19:48:33 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:48:33.559090 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486f5e9a-1bb6-42fa-8140-ecdc3ddce6d7 None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:48:33.655077 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486f5e9a-1bb6-42fa-8140-ecdc3ddce6d7 None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:48:33.688717 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8a0ea7-7099-45db-9875-93ad19efc3cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:48:33.689332 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8a0ea7-7099-45db-9875-93ad19efc3cc None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:48:33.689612 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8a0ea7-7099-45db-9875-93ad19efc3cc None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:48:33.689882 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8a0ea7-7099-45db-9875-93ad19efc3cc None admin] PATH_INFO: `/v3/roles` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:48:33.690594 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8a0ea7-7099-45db-9875-93ad19efc3cc None admin] RBAC: Adding query filter params (name=admin) {{(pid=80364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:48:33.690896 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8a0ea7-7099-45db-9875-93ad19efc3cc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:48:33.692271 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8a0ea7-7099-45db-9875-93ad19efc3cc None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:48:33.700614 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 24/50] 10.222.0.27 () {56 vars in 1099 bytes} [Fri Aug 5 19:48:33 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:48:33.706038 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed16eb3-aa3d-4999-b8f2-35ece39fa8bb None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:48:33.767965 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:48:33.807086 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed16eb3-aa3d-4999-b8f2-35ece39fa8bb None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:48:33.839154 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b5c15f-d427-419d-9647-8090ee8b48e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:48:33.839783 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b5c15f-d427-419d-9647-8090ee8b48e9 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:48:33.840062 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b5c15f-d427-419d-9647-8090ee8b48e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:48:33.840334 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b5c15f-d427-419d-9647-8090ee8b48e9 None admin] PATH_INFO: `/v3/users/03d1834a64cd4a43829827cb99a72747` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:48:33.841722 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b5c15f-d427-419d-9647-8090ee8b48e9 None admin] RBAC: Authorizing `identity:get_user(user_id=03d1834a64cd4a43829827cb99a72747)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:48:33.843178 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b5c15f-d427-419d-9647-8090ee8b48e9 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:48:33.844801 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 27/51] 10.222.0.27 () {56 vars in 1177 bytes} [Fri Aug 5 19:48:33 2022] GET /identity/v3/users/03d1834a64cd4a43829827cb99a72747 => generated 247 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:48:33.850716 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8a0ea7-7099-45db-9875-93ad19efc3cc None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:48:33.863858 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:48:33.951782 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8a0ea7-7099-45db-9875-93ad19efc3cc None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:48:33.987561 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2ebab4-dc88-490d-8724-e6f71e0e0544 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:48:33.988204 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2ebab4-dc88-490d-8724-e6f71e0e0544 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:48:33.988485 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2ebab4-dc88-490d-8724-e6f71e0e0544 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:48:33.988784 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2ebab4-dc88-490d-8724-e6f71e0e0544 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:48:33.989987 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f2ebab4-dc88-490d-8724-e6f71e0e0544 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:48:33.991426 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f2ebab4-dc88-490d-8724-e6f71e0e0544 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:48:33.993039 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 25/52] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Aug 5 19:48:33 2022] GET /identity/v3/domains/default => generated 199 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:48:33.998926 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b5c15f-d427-419d-9647-8090ee8b48e9 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:48:34.097239 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b5c15f-d427-419d-9647-8090ee8b48e9 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:48:34.105091 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5448dbc5-4f28-46f2-8434-656fbd1a1c2f None None] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:48:34.105392 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5448dbc5-4f28-46f2-8434-656fbd1a1c2f None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:48:34.105682 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5448dbc5-4f28-46f2-8434-656fbd1a1c2f None None] PATH_INFO: `/` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:48:34.107330 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 26/53] 10.222.0.27 () {54 vars in 927 bytes} [Fri Aug 5 19:48:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 05 19:48:34.112101 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8f1cbb-5612-4b91-af8b-0ec7c0c6e528 None None] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:48:34.112477 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8f1cbb-5612-4b91-af8b-0ec7c0c6e528 None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:48:34.112857 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8f1cbb-5612-4b91-af8b-0ec7c0c6e528 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:48:34.126742 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:48:34.130771 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.auth.core [None req-9c8f1cbb-5612-4b91-af8b-0ec7c0c6e528 None None] MFA Rules not processed for user `03d1834a64cd4a43829827cb99a72747`. Rule list: `[]` (Enabled: `True`). {{(pid=80364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:48:34.131984 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-9c8f1cbb-5612-4b91-af8b-0ec7c0c6e528 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:48:34.132139 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0e4b22-56c6-4bb2-a847-fb73b3b43caa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:48:34.132783 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0e4b22-56c6-4bb2-a847-fb73b3b43caa None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:48:34.133066 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0e4b22-56c6-4bb2-a847-fb73b3b43caa None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:48:34.133343 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0e4b22-56c6-4bb2-a847-fb73b3b43caa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:48:34.134136 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0e4b22-56c6-4bb2-a847-fb73b3b43caa None admin] RBAC: Adding query filter params (role.id=c173688a960b46e5be65f0ccd6fbd734, scope.domain.id=default, user.id=03d1834a64cd4a43829827cb99a72747) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:48:34.134445 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0e4b22-56c6-4bb2-a847-fb73b3b43caa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:48:34.135897 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0e4b22-56c6-4bb2-a847-fb73b3b43caa None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:48:34.144766 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 28/54] 10.222.0.27 () {56 vars in 1322 bytes} [Fri Aug 5 19:48:33 2022] GET /identity/v3/role_assignments?user.id=03d1834a64cd4a43829827cb99a72747&scope.domain.id=default&role.id=c173688a960b46e5be65f0ccd6fbd734 => generated 530 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:48:34.204102 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 27/55] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Aug 5 19:48:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 05 19:48:34.228713 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0e4b22-56c6-4bb2-a847-fb73b3b43caa None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:48:34.332297 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0e4b22-56c6-4bb2-a847-fb73b3b43caa None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:48:34.368410 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4541afa7-8efc-42fb-82ae-da033d732f05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:48:34.368864 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4541afa7-8efc-42fb-82ae-da033d732f05 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:48:34.369145 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4541afa7-8efc-42fb-82ae-da033d732f05 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:48:34.369418 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4541afa7-8efc-42fb-82ae-da033d732f05 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:48:34.370530 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4541afa7-8efc-42fb-82ae-da033d732f05 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:48:34.371991 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4541afa7-8efc-42fb-82ae-da033d732f05 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:48:34.373629 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 29/56] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Aug 5 19:48:34 2022] GET /identity/v3/domains/default => generated 199 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:48:34.379389 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8f1cbb-5612-4b91-af8b-0ec7c0c6e528 None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:48:34.389375 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:48:34.476400 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8f1cbb-5612-4b91-af8b-0ec7c0c6e528 None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:48:34.508737 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f43e561-ab4e-496b-a2ac-64ad946ddde7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:48:34.509350 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f43e561-ab4e-496b-a2ac-64ad946ddde7 None admin] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:48:34.509635 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f43e561-ab4e-496b-a2ac-64ad946ddde7 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:48:34.509914 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f43e561-ab4e-496b-a2ac-64ad946ddde7 None admin] PATH_INFO: `/v3/projects` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:48:34.510593 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f43e561-ab4e-496b-a2ac-64ad946ddde7 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:48:34.512038 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f43e561-ab4e-496b-a2ac-64ad946ddde7 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:48:34.537494 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 28/57] 10.222.0.27 () {60 vars in 1140 bytes} [Fri Aug 5 19:48:34 2022] POST /identity/v3/projects => generated 298 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 19:48:36.515347 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f12489-b4ba-4c0b-bbe6-90bf392f3338 None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:48:36.515850 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f12489-b4ba-4c0b-bbe6-90bf392f3338 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:48:36.515960 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f12489-b4ba-4c0b-bbe6-90bf392f3338 None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:48:36.517303 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 30/58] 10.222.0.27 () {54 vars in 927 bytes} [Fri Aug 5 19:48:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:48:36.522467 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f97b0a5-d9b0-4e7b-9abb-33042b2eb027 None None] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:48:36.522852 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f97b0a5-d9b0-4e7b-9abb-33042b2eb027 None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:48:36.523024 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f97b0a5-d9b0-4e7b-9abb-33042b2eb027 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:48:36.540723 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.auth.core [None req-6f97b0a5-d9b0-4e7b-9abb-33042b2eb027 None None] MFA Rules not processed for user `03d1834a64cd4a43829827cb99a72747`. Rule list: `[]` (Enabled: `True`). {{(pid=80364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:48:36.541893 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-6f97b0a5-d9b0-4e7b-9abb-33042b2eb027 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:48:36.595910 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:48:36.607808 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 29/59] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Aug 5 19:48:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 19:48:36.632355 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f12489-b4ba-4c0b-bbe6-90bf392f3338 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:48:36.700513 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:48:36.740003 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f12489-b4ba-4c0b-bbe6-90bf392f3338 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:48:36.771687 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd41314-da60-4759-b9bf-167fe5bcf9bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:48:36.772316 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd41314-da60-4759-b9bf-167fe5bcf9bf None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:48:36.772626 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd41314-da60-4759-b9bf-167fe5bcf9bf None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:48:36.772902 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd41314-da60-4759-b9bf-167fe5bcf9bf None admin] PATH_INFO: `/v3/domains/default` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:48:36.774048 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd41314-da60-4759-b9bf-167fe5bcf9bf None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:48:36.775748 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd41314-da60-4759-b9bf-167fe5bcf9bf None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:48:36.777360 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 31/60] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Aug 5 19:48:36 2022] GET /identity/v3/domains/default => generated 199 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:48:36.783633 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f97b0a5-d9b0-4e7b-9abb-33042b2eb027 None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:48:36.880106 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f97b0a5-d9b0-4e7b-9abb-33042b2eb027 None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:48:36.913694 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500a4d62-b683-40f9-97c6-62f0bee209c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:48:36.914244 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500a4d62-b683-40f9-97c6-62f0bee209c8 None admin] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:48:36.914527 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500a4d62-b683-40f9-97c6-62f0bee209c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:48:36.914806 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500a4d62-b683-40f9-97c6-62f0bee209c8 None admin] PATH_INFO: `/v3/users` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:48:36.915484 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500a4d62-b683-40f9-97c6-62f0bee209c8 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:48:36.917160 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500a4d62-b683-40f9-97c6-62f0bee209c8 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:48:36.943269 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 30/61] 10.222.0.27 () {60 vars in 1132 bytes} [Fri Aug 5 19:48:36 2022] POST /identity/v3/users => generated 275 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 19:48:40.094527 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80628914-e855-40ec-bb70-26e48815375e None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:48:40.095623 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80628914-e855-40ec-bb70-26e48815375e None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:48:40.096117 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80628914-e855-40ec-bb70-26e48815375e None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:48:40.098003 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 32/62] 10.222.0.27 () {54 vars in 927 bytes} [Fri Aug 5 19:48:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:48:40.103302 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f01cb22-8ed7-4150-b9f6-3f04379c21f7 None None] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:48:40.104105 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f01cb22-8ed7-4150-b9f6-3f04379c21f7 None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:48:40.104645 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f01cb22-8ed7-4150-b9f6-3f04379c21f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:48:40.125487 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.auth.core [None req-3f01cb22-8ed7-4150-b9f6-3f04379c21f7 None None] MFA Rules not processed for user `03d1834a64cd4a43829827cb99a72747`. Rule list: `[]` (Enabled: `True`). {{(pid=80364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:48:40.127127 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-3f01cb22-8ed7-4150-b9f6-3f04379c21f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:48:40.203026 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:48:40.231086 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 31/63] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Aug 5 19:48:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 19:48:40.274021 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80628914-e855-40ec-bb70-26e48815375e None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:48:40.286239 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3001b31-d2eb-49ee-83c5-64ebbcd23ca4 None None] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:48:40.286832 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3001b31-d2eb-49ee-83c5-64ebbcd23ca4 None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:48:40.287305 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3001b31-d2eb-49ee-83c5-64ebbcd23ca4 None None] PATH_INFO: `/` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:48:40.288903 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 32/64] 10.222.0.27 () {54 vars in 927 bytes} [Fri Aug 5 19:48:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:48:40.294422 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80036cb7-fb08-4f5d-b31e-cfe1233646c2 None None] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:48:40.294704 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80036cb7-fb08-4f5d-b31e-cfe1233646c2 None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:48:40.295101 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80036cb7-fb08-4f5d-b31e-cfe1233646c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:48:40.318031 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.auth.core [None req-80036cb7-fb08-4f5d-b31e-cfe1233646c2 None None] MFA Rules not processed for user `03d1834a64cd4a43829827cb99a72747`. Rule list: `[]` (Enabled: `True`). {{(pid=80364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:48:40.319313 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-80036cb7-fb08-4f5d-b31e-cfe1233646c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:48:40.398162 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80628914-e855-40ec-bb70-26e48815375e None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:48:40.415639 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 33/65] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Aug 5 19:48:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 05 19:48:40.417461 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99c1336-3ccc-438a-9e15-496ed58a4355 None None] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:48:40.417972 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99c1336-3ccc-438a-9e15-496ed58a4355 None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:48:40.418451 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99c1336-3ccc-438a-9e15-496ed58a4355 None None] PATH_INFO: `/` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:48:40.424346 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 34/66] 10.222.0.27 () {54 vars in 927 bytes} [Fri Aug 5 19:48:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:48:40.426146 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979e28f1-22e7-467d-b2ab-47510193c479 None None] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:48:40.426660 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979e28f1-22e7-467d-b2ab-47510193c479 None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:48:40.427141 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979e28f1-22e7-467d-b2ab-47510193c479 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:48:40.448547 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.auth.core [None req-979e28f1-22e7-467d-b2ab-47510193c479 None None] MFA Rules not processed for user `03d1834a64cd4a43829827cb99a72747`. Rule list: `[]` (Enabled: `True`). {{(pid=80364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:48:40.451096 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:48:40.452027 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-979e28f1-22e7-467d-b2ab-47510193c479 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:48:40.457436 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5ab2ae-0204-4178-b07f-f87a46950a90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:48:40.458180 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5ab2ae-0204-4178-b07f-f87a46950a90 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:48:40.458636 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5ab2ae-0204-4178-b07f-f87a46950a90 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:48:40.459094 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5ab2ae-0204-4178-b07f-f87a46950a90 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:48:40.471889 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c5ab2ae-0204-4178-b07f-f87a46950a90 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 19:48:40.478059 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-3c5ab2ae-0204-4178-b07f-f87a46950a90 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 05 19:48:40.478059 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:48:40.478059 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:48:40.478059 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:48:40.478059 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:48:40.478059 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:48:40.478059 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:48:40.478059 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:48:40.478059 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:48:40.478059 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:48:40.478059 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:48:40.478059 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:48:40.478059 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 19:48:40.478059 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 19:48:40.478059 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 19:48:40.478059 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 19:48:40.478971 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 19:48:40.478971 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 19:48:40.478971 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 19:48:40.478971 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 19:48:40.478971 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:48:40.478971 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:48:40.478971 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:48:40.478971 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:48:40.478971 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:48:40.478971 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:48:40.478971 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:48:40.478971 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Aug 05 19:48:40.478971 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:48:40.478971 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:48:40.478971 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:48:40.478971 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:48:40.479756 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:48:40.479756 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:48:40.479756 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:48:40.479756 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:48:40.479756 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 19:48:40.479756 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 19:48:40.479756 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 19:48:40.479756 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 19:48:40.479756 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 19:48:40.479756 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 19:48:40.479756 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 05 19:48:40.479756 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Aug 05 19:48:40.482294 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 33/67] 10.222.0.27 () {56 vars in 1099 bytes} [Fri Aug 5 19:48:40 2022] GET /identity/v3/roles/member => generated 84 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:48:40.483300 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5ab2ae-0204-4178-b07f-f87a46950a90 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:48:40.547920 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 35/68] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Aug 5 19:48:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 19:48:40.549925 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979e28f1-22e7-467d-b2ab-47510193c479 None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:48:40.597057 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:48:40.612695 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5ab2ae-0204-4178-b07f-f87a46950a90 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:48:40.648050 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8746d03-2017-4c6f-9916-f3d0b6cef8fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:48:40.648657 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8746d03-2017-4c6f-9916-f3d0b6cef8fa None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:48:40.648956 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8746d03-2017-4c6f-9916-f3d0b6cef8fa None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:48:40.649232 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8746d03-2017-4c6f-9916-f3d0b6cef8fa None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:48:40.661237 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8746d03-2017-4c6f-9916-f3d0b6cef8fa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 19:48:40.668056 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-e8746d03-2017-4c6f-9916-f3d0b6cef8fa None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 05 19:48:40.668056 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:48:40.668056 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:48:40.668056 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:48:40.668056 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:48:40.668056 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:48:40.668056 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:48:40.668056 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:48:40.668056 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:48:40.668056 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:48:40.668056 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:48:40.668056 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:48:40.668056 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 19:48:40.668056 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 19:48:40.668056 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 19:48:40.668056 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 19:48:40.669018 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 19:48:40.669018 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 19:48:40.669018 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 19:48:40.669018 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 19:48:40.669018 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:48:40.669018 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:48:40.669018 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:48:40.669018 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:48:40.669018 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:48:40.669018 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:48:40.669018 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:48:40.669018 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Aug 05 19:48:40.669018 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:48:40.669018 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:48:40.669018 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:48:40.669018 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:48:40.669778 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:48:40.669778 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:48:40.669778 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:48:40.669778 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:48:40.669778 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 19:48:40.669778 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 19:48:40.669778 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 19:48:40.669778 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 19:48:40.669778 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 19:48:40.669778 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 19:48:40.669778 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 05 19:48:40.669778 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Aug 05 19:48:40.670348 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 34/69] 10.222.0.27 () {56 vars in 1114 bytes} [Fri Aug 5 19:48:40 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:48:40.671196 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8746d03-2017-4c6f-9916-f3d0b6cef8fa None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:48:40.685521 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979e28f1-22e7-467d-b2ab-47510193c479 None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:48:40.721049 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38c7c0b-4817-437e-b555-9e417b387de1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:48:40.721671 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38c7c0b-4817-437e-b555-9e417b387de1 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:48:40.721958 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38c7c0b-4817-437e-b555-9e417b387de1 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:48:40.722407 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38c7c0b-4817-437e-b555-9e417b387de1 None admin] PATH_INFO: `/v3/roles` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:48:40.724325 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b38c7c0b-4817-437e-b555-9e417b387de1 None admin] RBAC: Adding query filter params (name=member) {{(pid=80364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:48:40.724662 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b38c7c0b-4817-437e-b555-9e417b387de1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:48:40.727160 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b38c7c0b-4817-437e-b555-9e417b387de1 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:48:40.729234 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:48:40.736162 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 36/70] 10.222.0.27 () {56 vars in 1101 bytes} [Fri Aug 5 19:48:40 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:48:40.737510 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18f3d39-4db7-4f16-894a-71e1a27b8f19 None None] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:48:40.737808 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18f3d39-4db7-4f16-894a-71e1a27b8f19 None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:48:40.738088 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18f3d39-4db7-4f16-894a-71e1a27b8f19 None None] PATH_INFO: `/` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:48:40.739361 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 37/71] 10.222.0.27 () {54 vars in 927 bytes} [Fri Aug 5 19:48:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:48:40.740061 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18f3d39-4db7-4f16-894a-71e1a27b8f19 None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:48:40.779472 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8746d03-2017-4c6f-9916-f3d0b6cef8fa None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:48:40.813678 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74951abc-d1ed-4fc4-b878-1d333d0ba75b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:48:40.814319 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74951abc-d1ed-4fc4-b878-1d333d0ba75b None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:48:40.814604 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74951abc-d1ed-4fc4-b878-1d333d0ba75b None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:48:40.814900 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74951abc-d1ed-4fc4-b878-1d333d0ba75b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:48:40.824841 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74951abc-d1ed-4fc4-b878-1d333d0ba75b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 19:48:40.830689 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-74951abc-d1ed-4fc4-b878-1d333d0ba75b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 19:48:40.830689 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:48:40.830689 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:48:40.830689 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:48:40.830689 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:48:40.830689 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:48:40.830689 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:48:40.830689 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:48:40.830689 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:48:40.830689 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:48:40.830689 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:48:40.830689 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:48:40.830689 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 19:48:40.830689 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 19:48:40.830689 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 19:48:40.830689 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 19:48:40.831536 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 19:48:40.831536 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 19:48:40.831536 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 19:48:40.831536 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 19:48:40.831536 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:48:40.831536 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:48:40.831536 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:48:40.831536 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:48:40.831536 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:48:40.831536 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:48:40.831536 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:48:40.831536 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Aug 05 19:48:40.831536 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:48:40.831536 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:48:40.831536 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:48:40.831536 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:48:40.832431 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:48:40.832431 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:48:40.832431 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:48:40.832431 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:48:40.832431 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 19:48:40.832431 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 19:48:40.832431 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 19:48:40.832431 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 19:48:40.832431 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 19:48:40.832431 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 19:48:40.832431 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 19:48:40.832431 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Aug 05 19:48:40.832431 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 35/72] 10.222.0.27 () {56 vars in 1096 bytes} [Fri Aug 5 19:48:40 2022] GET /identity/v3/roles/admin => generated 83 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:48:40.833371 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57e5da9-c968-4309-b7f4-b9013de3a0ad None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:48:40.833371 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57e5da9-c968-4309-b7f4-b9013de3a0ad None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:48:40.833598 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57e5da9-c968-4309-b7f4-b9013de3a0ad None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:48:40.834853 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 36/73] 10.222.0.27 () {54 vars in 927 bytes} [Fri Aug 5 19:48:40 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:48:40.835551 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57e5da9-c968-4309-b7f4-b9013de3a0ad None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:48:40.844395 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:48:40.849862 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18f3d39-4db7-4f16-894a-71e1a27b8f19 None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:48:40.883648 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98bc54a-fba1-4719-aaa4-f9d9a62d18c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:48:40.884269 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98bc54a-fba1-4719-aaa4-f9d9a62d18c5 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:48:40.884578 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98bc54a-fba1-4719-aaa4-f9d9a62d18c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:48:40.884859 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98bc54a-fba1-4719-aaa4-f9d9a62d18c5 None admin] PATH_INFO: `/v3/roles` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:48:40.885604 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98bc54a-fba1-4719-aaa4-f9d9a62d18c5 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:48:40.885909 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98bc54a-fba1-4719-aaa4-f9d9a62d18c5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:48:40.887307 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98bc54a-fba1-4719-aaa4-f9d9a62d18c5 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:48:40.896256 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 38/74] 10.222.0.27 () {56 vars in 1111 bytes} [Fri Aug 5 19:48:40 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:48:40.897451 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e133e359-8795-4eaf-88ab-56c5d3e60fc4 None None] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:48:40.897740 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e133e359-8795-4eaf-88ab-56c5d3e60fc4 None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:48:40.898016 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e133e359-8795-4eaf-88ab-56c5d3e60fc4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:48:40.915301 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.auth.core [None req-e133e359-8795-4eaf-88ab-56c5d3e60fc4 None None] MFA Rules not processed for user `03d1834a64cd4a43829827cb99a72747`. Rule list: `[]` (Enabled: `True`). {{(pid=80364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:48:40.916420 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-e133e359-8795-4eaf-88ab-56c5d3e60fc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:48:40.940033 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57e5da9-c968-4309-b7f4-b9013de3a0ad None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:48:40.952678 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:48:40.973360 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08826f16-c8d9-436b-9ec4-e8c2bcfbb64a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:48:40.974004 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08826f16-c8d9-436b-9ec4-e8c2bcfbb64a None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:48:40.974284 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08826f16-c8d9-436b-9ec4-e8c2bcfbb64a None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:48:40.974565 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08826f16-c8d9-436b-9ec4-e8c2bcfbb64a None admin] PATH_INFO: `/v3/users/311403d2ab974eddb6f0bc18bcb7b8e0` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:48:40.981654 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 39/75] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Aug 5 19:48:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 19:48:40.982296 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e133e359-8795-4eaf-88ab-56c5d3e60fc4 None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:48:41.000894 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08826f16-c8d9-436b-9ec4-e8c2bcfbb64a None admin] RBAC: Authorizing `identity:get_user(user_id=311403d2ab974eddb6f0bc18bcb7b8e0)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:48:41.002939 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08826f16-c8d9-436b-9ec4-e8c2bcfbb64a None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:48:41.004674 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 37/76] 10.222.0.27 () {56 vars in 1177 bytes} [Fri Aug 5 19:48:40 2022] GET /identity/v3/users/311403d2ab974eddb6f0bc18bcb7b8e0 => generated 275 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:48:41.009176 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21df2db2-cece-47c4-bafe-28f73be76ba8 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:48:41.009176 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21df2db2-cece-47c4-bafe-28f73be76ba8 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:48:41.009176 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21df2db2-cece-47c4-bafe-28f73be76ba8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:48:41.025208 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-21df2db2-cece-47c4-bafe-28f73be76ba8 None None] MFA Rules not processed for user `03d1834a64cd4a43829827cb99a72747`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:48:41.026411 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-21df2db2-cece-47c4-bafe-28f73be76ba8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:48:41.077996 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:48:41.089335 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e133e359-8795-4eaf-88ab-56c5d3e60fc4 None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:48:41.093336 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 38/77] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Aug 5 19:48:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 19:48:41.095441 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21df2db2-cece-47c4-bafe-28f73be76ba8 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:48:41.138082 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6cb1fd-171b-4682-a531-6bf6b88b5701 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:48:41.138705 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6cb1fd-171b-4682-a531-6bf6b88b5701 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:48:41.138990 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6cb1fd-171b-4682-a531-6bf6b88b5701 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:48:41.139270 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6cb1fd-171b-4682-a531-6bf6b88b5701 None admin] PATH_INFO: `/v3/roles` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:48:41.139988 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff6cb1fd-171b-4682-a531-6bf6b88b5701 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:48:41.140299 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff6cb1fd-171b-4682-a531-6bf6b88b5701 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:48:41.142236 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff6cb1fd-171b-4682-a531-6bf6b88b5701 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:48:41.150565 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 40/78] 10.222.0.27 () {56 vars in 1099 bytes} [Fri Aug 5 19:48:40 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:48:41.151347 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6cb1fd-171b-4682-a531-6bf6b88b5701 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:48:41.197344 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:48:41.200655 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21df2db2-cece-47c4-bafe-28f73be76ba8 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:48:41.235811 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7590c135-4a92-4523-a3c4-b5c54849c587 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:48:41.236460 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7590c135-4a92-4523-a3c4-b5c54849c587 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:48:41.236769 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7590c135-4a92-4523-a3c4-b5c54849c587 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:48:41.237048 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7590c135-4a92-4523-a3c4-b5c54849c587 None admin] PATH_INFO: `/v3/users/311403d2ab974eddb6f0bc18bcb7b8e0` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:48:41.238244 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7590c135-4a92-4523-a3c4-b5c54849c587 None admin] RBAC: Authorizing `identity:get_user(user_id=311403d2ab974eddb6f0bc18bcb7b8e0)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:48:41.239715 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7590c135-4a92-4523-a3c4-b5c54849c587 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:48:41.241352 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 39/79] 10.222.0.27 () {56 vars in 1177 bytes} [Fri Aug 5 19:48:41 2022] GET /identity/v3/users/311403d2ab974eddb6f0bc18bcb7b8e0 => generated 275 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:48:41.246166 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7590c135-4a92-4523-a3c4-b5c54849c587 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:48:41.253235 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6cb1fd-171b-4682-a531-6bf6b88b5701 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:48:41.286616 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5017a215-3340-4c8d-a06b-7571b60b2007 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:48:41.287257 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5017a215-3340-4c8d-a06b-7571b60b2007 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:48:41.287537 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5017a215-3340-4c8d-a06b-7571b60b2007 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:48:41.287811 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5017a215-3340-4c8d-a06b-7571b60b2007 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:48:41.289022 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5017a215-3340-4c8d-a06b-7571b60b2007 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:48:41.290454 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5017a215-3340-4c8d-a06b-7571b60b2007 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:48:41.292195 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 41/80] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Aug 5 19:48:41 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:48:41.292804 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5017a215-3340-4c8d-a06b-7571b60b2007 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:48:41.314170 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:48:41.343668 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7590c135-4a92-4523-a3c4-b5c54849c587 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:48:41.378248 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2adae579-80ae-442a-b690-ae34ee074afb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:48:41.378876 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2adae579-80ae-442a-b690-ae34ee074afb None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:48:41.379163 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2adae579-80ae-442a-b690-ae34ee074afb None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:48:41.379440 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2adae579-80ae-442a-b690-ae34ee074afb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:48:41.396818 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5017a215-3340-4c8d-a06b-7571b60b2007 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:48:41.414089 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:48:41.430404 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae17ecab-0351-4586-936a-99582dcf652a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:48:41.431051 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae17ecab-0351-4586-936a-99582dcf652a None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:48:41.431339 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae17ecab-0351-4586-936a-99582dcf652a None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:48:41.431621 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae17ecab-0351-4586-936a-99582dcf652a None admin] PATH_INFO: `/v3/roles/member` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:48:41.442251 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae17ecab-0351-4586-936a-99582dcf652a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 19:48:41.447691 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application [None req-ae17ecab-0351-4586-936a-99582dcf652a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 05 19:48:41.447691 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:48:41.447691 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:48:41.447691 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:48:41.447691 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:48:41.447691 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:48:41.447691 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:48:41.447691 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:48:41.447691 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:48:41.447691 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:48:41.447691 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:48:41.447691 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:48:41.447691 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 19:48:41.447691 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 19:48:41.447691 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 19:48:41.447691 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 19:48:41.448556 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 19:48:41.448556 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 19:48:41.448556 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 19:48:41.448556 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 19:48:41.448556 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:48:41.448556 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:48:41.448556 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:48:41.448556 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:48:41.448556 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:48:41.448556 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:48:41.448556 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:48:41.448556 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application with Lock( Aug 05 19:48:41.448556 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:48:41.448556 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:48:41.448556 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:48:41.448556 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:48:41.449347 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:48:41.449347 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:48:41.449347 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:48:41.449347 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:48:41.449347 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 19:48:41.449347 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 19:48:41.449347 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 19:48:41.449347 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 19:48:41.449347 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 19:48:41.449347 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 19:48:41.449347 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 05 19:48:41.449347 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Aug 05 19:48:41.449347 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 42/81] 10.222.0.27 () {56 vars in 1099 bytes} [Fri Aug 5 19:48:41 2022] GET /identity/v3/roles/member => generated 84 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:48:41.449966 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae17ecab-0351-4586-936a-99582dcf652a None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:48:41.458984 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2adae579-80ae-442a-b690-ae34ee074afb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:48:41.460509 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2adae579-80ae-442a-b690-ae34ee074afb None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:48:41.532052 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:48:41.552000 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae17ecab-0351-4586-936a-99582dcf652a None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:48:41.560307 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 40/82] 10.222.0.27 () {58 vars in 1282 bytes} [Fri Aug 5 19:48:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 05 19:48:41.561840 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2adae579-80ae-442a-b690-ae34ee074afb None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:48:41.585039 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86b97e6-ca84-4682-995f-afef3797e998 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:48:41.585682 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86b97e6-ca84-4682-995f-afef3797e998 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:48:41.585959 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86b97e6-ca84-4682-995f-afef3797e998 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:48:41.586233 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86b97e6-ca84-4682-995f-afef3797e998 None admin] PATH_INFO: `/v3/users/03d1834a64cd4a43829827cb99a72747` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:48:41.587386 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e86b97e6-ca84-4682-995f-afef3797e998 None admin] RBAC: Authorizing `identity:get_user(user_id=03d1834a64cd4a43829827cb99a72747)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:48:41.588865 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e86b97e6-ca84-4682-995f-afef3797e998 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:48:41.590543 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 43/83] 10.222.0.27 () {56 vars in 1177 bytes} [Fri Aug 5 19:48:41 2022] GET /identity/v3/users/03d1834a64cd4a43829827cb99a72747 => generated 247 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:48:41.591269 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86b97e6-ca84-4682-995f-afef3797e998 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:48:41.630146 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:48:41.664427 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2adae579-80ae-442a-b690-ae34ee074afb None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:48:41.689604 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86b97e6-ca84-4682-995f-afef3797e998 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:48:41.699038 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887bb157-afc3-4857-a233-513f3ddec346 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:48:41.699594 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887bb157-afc3-4857-a233-513f3ddec346 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:48:41.699889 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887bb157-afc3-4857-a233-513f3ddec346 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:48:41.700163 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887bb157-afc3-4857-a233-513f3ddec346 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:48:41.722672 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2daea37e-e041-4b63-884c-f770c5d10a2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:48:41.723282 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2daea37e-e041-4b63-884c-f770c5d10a2d None admin] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:48:41.723566 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2daea37e-e041-4b63-884c-f770c5d10a2d None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:48:41.723848 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2daea37e-e041-4b63-884c-f770c5d10a2d None admin] PATH_INFO: `/v3/users` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:48:41.724554 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2daea37e-e041-4b63-884c-f770c5d10a2d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:48:41.726010 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2daea37e-e041-4b63-884c-f770c5d10a2d None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:48:41.746891 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 44/84] 10.222.0.27 () {60 vars in 1132 bytes} [Fri Aug 5 19:48:41 2022] POST /identity/v3/users => generated 289 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 19:48:41.747745 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2daea37e-e041-4b63-884c-f770c5d10a2d None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:48:41.765059 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:48:41.772023 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887bb157-afc3-4857-a233-513f3ddec346 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:48:41.773581 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887bb157-afc3-4857-a233-513f3ddec346 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:48:41.857396 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:48:41.878817 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2daea37e-e041-4b63-884c-f770c5d10a2d None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:48:41.909391 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 41/85] 10.222.0.27 () {58 vars in 1282 bytes} [Fri Aug 5 19:48:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 05 19:48:41.910335 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887bb157-afc3-4857-a233-513f3ddec346 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:48:41.924540 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e62584f-e1b4-4c3e-8e41-e3262a9c6617 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:48:41.924898 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e62584f-e1b4-4c3e-8e41-e3262a9c6617 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:48:41.925179 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e62584f-e1b4-4c3e-8e41-e3262a9c6617 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:48:41.925454 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e62584f-e1b4-4c3e-8e41-e3262a9c6617 None admin] PATH_INFO: `/v3/roles` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:48:41.926177 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e62584f-e1b4-4c3e-8e41-e3262a9c6617 None admin] RBAC: Adding query filter params (name=member) {{(pid=80364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:48:41.926481 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e62584f-e1b4-4c3e-8e41-e3262a9c6617 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:48:41.927882 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e62584f-e1b4-4c3e-8e41-e3262a9c6617 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:48:41.937328 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 45/86] 10.222.0.27 () {56 vars in 1101 bytes} [Fri Aug 5 19:48:41 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:48:41.938246 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e62584f-e1b4-4c3e-8e41-e3262a9c6617 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:48:42.018299 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:48:42.034870 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887bb157-afc3-4857-a233-513f3ddec346 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:48:42.065764 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e62584f-e1b4-4c3e-8e41-e3262a9c6617 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:48:42.075842 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2753fd-93da-439b-a286-78cbfe692745 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:48:42.076534 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2753fd-93da-439b-a286-78cbfe692745 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:48:42.076823 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2753fd-93da-439b-a286-78cbfe692745 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:48:42.077098 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2753fd-93da-439b-a286-78cbfe692745 None admin] PATH_INFO: `/v3/projects/6f67d76798fb469c8ec9dd2d185572c3` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:48:42.078279 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e2753fd-93da-439b-a286-78cbfe692745 None admin] RBAC: Authorizing `identity:get_project(project_id=6f67d76798fb469c8ec9dd2d185572c3)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:48:42.079815 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e2753fd-93da-439b-a286-78cbfe692745 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:48:42.081709 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 42/87] 10.222.0.27 () {56 vars in 1186 bytes} [Fri Aug 5 19:48:41 2022] GET /identity/v3/projects/6f67d76798fb469c8ec9dd2d185572c3 => generated 298 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:48:42.082844 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2753fd-93da-439b-a286-78cbfe692745 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:48:42.115067 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb76213-9d39-42a4-8ee1-9d4b50c5dbf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:48:42.116245 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb76213-9d39-42a4-8ee1-9d4b50c5dbf7 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:48:42.117136 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb76213-9d39-42a4-8ee1-9d4b50c5dbf7 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:48:42.117898 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb76213-9d39-42a4-8ee1-9d4b50c5dbf7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:48:42.121302 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:48:42.201137 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2753fd-93da-439b-a286-78cbfe692745 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:48:42.207733 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb76213-9d39-42a4-8ee1-9d4b50c5dbf7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:48:42.209769 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb76213-9d39-42a4-8ee1-9d4b50c5dbf7 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:48:42.240419 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c457e474-c2c4-4eb6-82fb-df0cba100d76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:48:42.241135 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c457e474-c2c4-4eb6-82fb-df0cba100d76 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:48:42.241419 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c457e474-c2c4-4eb6-82fb-df0cba100d76 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:48:42.241697 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c457e474-c2c4-4eb6-82fb-df0cba100d76 None admin] PATH_INFO: `/v3/projects/6f67d76798fb469c8ec9dd2d185572c3` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:48:42.243094 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c457e474-c2c4-4eb6-82fb-df0cba100d76 None admin] RBAC: Authorizing `identity:get_project(project_id=6f67d76798fb469c8ec9dd2d185572c3)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:48:42.244326 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c457e474-c2c4-4eb6-82fb-df0cba100d76 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:48:42.246232 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 43/88] 10.222.0.27 () {56 vars in 1186 bytes} [Fri Aug 5 19:48:42 2022] GET /identity/v3/projects/6f67d76798fb469c8ec9dd2d185572c3 => generated 298 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:48:42.247213 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c457e474-c2c4-4eb6-82fb-df0cba100d76 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:48:42.291513 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:48:42.333609 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 46/89] 10.222.0.27 () {58 vars in 1282 bytes} [Fri Aug 5 19:48:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 391 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 05 19:48:42.333609 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb76213-9d39-42a4-8ee1-9d4b50c5dbf7 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:48:42.378144 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c457e474-c2c4-4eb6-82fb-df0cba100d76 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:48:42.381581 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:48:42.422510 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d8ac4e-fcf5-4cc3-918b-4afcd25ae8cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:48:42.423713 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d8ac4e-fcf5-4cc3-918b-4afcd25ae8cb None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:48:42.423994 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d8ac4e-fcf5-4cc3-918b-4afcd25ae8cb None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:48:42.424268 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d8ac4e-fcf5-4cc3-918b-4afcd25ae8cb None admin] PATH_INFO: `/v3/users/311403d2ab974eddb6f0bc18bcb7b8e0` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:48:42.426033 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d8ac4e-fcf5-4cc3-918b-4afcd25ae8cb None admin] RBAC: Authorizing `identity:get_user(user_id=311403d2ab974eddb6f0bc18bcb7b8e0)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:48:42.427948 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d8ac4e-fcf5-4cc3-918b-4afcd25ae8cb None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:48:42.430520 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 44/90] 10.222.0.27 () {56 vars in 1177 bytes} [Fri Aug 5 19:48:42 2022] GET /identity/v3/users/311403d2ab974eddb6f0bc18bcb7b8e0 => generated 275 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:48:42.431574 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d8ac4e-fcf5-4cc3-918b-4afcd25ae8cb None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:48:42.453360 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb76213-9d39-42a4-8ee1-9d4b50c5dbf7 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:48:42.490601 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ad7e95-f915-4039-b48d-fc87509a08c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:48:42.491714 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ad7e95-f915-4039-b48d-fc87509a08c6 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:48:42.492216 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ad7e95-f915-4039-b48d-fc87509a08c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:48:42.492732 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ad7e95-f915-4039-b48d-fc87509a08c6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:48:42.493726 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ad7e95-f915-4039-b48d-fc87509a08c6 None admin] RBAC: Adding query filter params (role.id=1a4f694d53054a24b580138b1ba77f3c, scope.project.id=6f67d76798fb469c8ec9dd2d185572c3, user.id=311403d2ab974eddb6f0bc18bcb7b8e0) {{(pid=80364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:48:42.494498 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ad7e95-f915-4039-b48d-fc87509a08c6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:48:42.497209 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ad7e95-f915-4039-b48d-fc87509a08c6 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:48:42.504746 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 47/91] 10.222.0.27 () {56 vars in 1374 bytes} [Fri Aug 5 19:48:42 2022] GET /identity/v3/role_assignments?user.id=311403d2ab974eddb6f0bc18bcb7b8e0&scope.project.id=6f67d76798fb469c8ec9dd2d185572c3&role.id=1a4f694d53054a24b580138b1ba77f3c => generated 259 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:48:42.506341 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ad7e95-f915-4039-b48d-fc87509a08c6 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:48:42.550742 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:48:42.613604 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d8ac4e-fcf5-4cc3-918b-4afcd25ae8cb None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:48:42.643198 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ad7e95-f915-4039-b48d-fc87509a08c6 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:48:42.686451 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3535549-5643-4db3-9e0b-09122d882c0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:48:42.687447 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3535549-5643-4db3-9e0b-09122d882c0f None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:48:42.687923 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3535549-5643-4db3-9e0b-09122d882c0f None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:48:42.688955 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3535549-5643-4db3-9e0b-09122d882c0f None admin] PATH_INFO: `/v3/projects/6f67d76798fb469c8ec9dd2d185572c3` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:48:42.690332 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3535549-5643-4db3-9e0b-09122d882c0f None admin] RBAC: Authorizing `identity:get_project(project_id=6f67d76798fb469c8ec9dd2d185572c3)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:48:42.690434 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63d63b2-1427-4265-9a8c-b06280ebc9a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:48:42.690962 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63d63b2-1427-4265-9a8c-b06280ebc9a6 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:48:42.691245 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63d63b2-1427-4265-9a8c-b06280ebc9a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:48:42.691522 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63d63b2-1427-4265-9a8c-b06280ebc9a6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:48:42.692325 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d63d63b2-1427-4265-9a8c-b06280ebc9a6 None admin] RBAC: Adding query filter params (role.id=dde0aa8028794132913bcba4d91858b8, scope.project.id=6f67d76798fb469c8ec9dd2d185572c3, user.id=311403d2ab974eddb6f0bc18bcb7b8e0) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:48:42.692779 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d63d63b2-1427-4265-9a8c-b06280ebc9a6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:48:42.693170 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3535549-5643-4db3-9e0b-09122d882c0f None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:48:42.694216 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d63d63b2-1427-4265-9a8c-b06280ebc9a6 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:48:42.695259 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 48/92] 10.222.0.27 () {56 vars in 1186 bytes} [Fri Aug 5 19:48:42 2022] GET /identity/v3/projects/6f67d76798fb469c8ec9dd2d185572c3 => generated 298 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:48:42.696738 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3535549-5643-4db3-9e0b-09122d882c0f None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:48:42.714478 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 45/93] 10.222.0.27 () {56 vars in 1374 bytes} [Fri Aug 5 19:48:42 2022] GET /identity/v3/role_assignments?user.id=311403d2ab974eddb6f0bc18bcb7b8e0&scope.project.id=6f67d76798fb469c8ec9dd2d185572c3&role.id=dde0aa8028794132913bcba4d91858b8 => generated 259 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:48:42.715187 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63d63b2-1427-4265-9a8c-b06280ebc9a6 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:48:42.784931 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:48:42.818016 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:48:42.824153 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3535549-5643-4db3-9e0b-09122d882c0f None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:48:42.862526 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f7ff7e-c188-4f08-b61d-d73502750969 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:48:42.863615 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f7ff7e-c188-4f08-b61d-d73502750969 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:48:42.864092 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f7ff7e-c188-4f08-b61d-d73502750969 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:48:42.864676 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f7ff7e-c188-4f08-b61d-d73502750969 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:48:42.898643 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63d63b2-1427-4265-9a8c-b06280ebc9a6 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:48:42.942823 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f7ff7e-c188-4f08-b61d-d73502750969 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:48:42.943147 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe4415d-0816-401e-8c96-86f80614b3ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:48:42.943717 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe4415d-0816-401e-8c96-86f80614b3ac None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:48:42.944005 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe4415d-0816-401e-8c96-86f80614b3ac None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:48:42.944281 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe4415d-0816-401e-8c96-86f80614b3ac None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:48:42.945116 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe4415d-0816-401e-8c96-86f80614b3ac None admin] RBAC: Adding query filter params (role.id=c173688a960b46e5be65f0ccd6fbd734, scope.project.id=6f67d76798fb469c8ec9dd2d185572c3, user.id=03d1834a64cd4a43829827cb99a72747) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:48:42.945427 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe4415d-0816-401e-8c96-86f80614b3ac None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:48:42.946050 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f7ff7e-c188-4f08-b61d-d73502750969 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:48:42.946878 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe4415d-0816-401e-8c96-86f80614b3ac None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:48:42.953166 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 46/94] 10.222.0.27 () {56 vars in 1374 bytes} [Fri Aug 5 19:48:42 2022] GET /identity/v3/role_assignments?user.id=03d1834a64cd4a43829827cb99a72747&scope.project.id=6f67d76798fb469c8ec9dd2d185572c3&role.id=c173688a960b46e5be65f0ccd6fbd734 => generated 259 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:48:43.053992 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:48:43.059335 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 49/95] 10.222.0.27 () {58 vars in 1282 bytes} [Fri Aug 5 19:48:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 05 19:48:43.069768 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe4415d-0816-401e-8c96-86f80614b3ac None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:48:43.204859 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe4415d-0816-401e-8c96-86f80614b3ac None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:48:43.207600 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:48:43.254386 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad57e968-f6ed-4f3b-be13-afd48427a913 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:48:43.255410 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad57e968-f6ed-4f3b-be13-afd48427a913 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:48:43.255894 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad57e968-f6ed-4f3b-be13-afd48427a913 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:48:43.256365 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad57e968-f6ed-4f3b-be13-afd48427a913 None admin] PATH_INFO: `/v3/projects/7ac0f0a66e994494baf51533717498da` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:48:43.257813 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad57e968-f6ed-4f3b-be13-afd48427a913 None admin] RBAC: Authorizing `identity:get_project(project_id=7ac0f0a66e994494baf51533717498da)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:48:43.259462 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad57e968-f6ed-4f3b-be13-afd48427a913 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:48:43.261403 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 47/96] 10.222.0.27 () {56 vars in 1186 bytes} [Fri Aug 5 19:48:43 2022] GET /identity/v3/projects/7ac0f0a66e994494baf51533717498da => generated 312 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:48:43.268608 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f7ff7e-c188-4f08-b61d-d73502750969 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:48:43.414062 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f7ff7e-c188-4f08-b61d-d73502750969 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:48:43.451003 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0109ed50-1596-47f8-b75b-0a8fccadb4ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:48:43.451624 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0109ed50-1596-47f8-b75b-0a8fccadb4ce None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:48:43.451905 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0109ed50-1596-47f8-b75b-0a8fccadb4ce None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:48:43.452177 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0109ed50-1596-47f8-b75b-0a8fccadb4ce None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:48:43.453006 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0109ed50-1596-47f8-b75b-0a8fccadb4ce None admin] RBAC: Adding query filter params (role.id=1a4f694d53054a24b580138b1ba77f3c, scope.project.id=7ac0f0a66e994494baf51533717498da, user.id=311403d2ab974eddb6f0bc18bcb7b8e0) {{(pid=80364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:48:43.453316 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0109ed50-1596-47f8-b75b-0a8fccadb4ce None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:48:43.455387 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0109ed50-1596-47f8-b75b-0a8fccadb4ce None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:48:43.464679 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 50/97] 10.222.0.27 () {56 vars in 1374 bytes} [Fri Aug 5 19:48:43 2022] GET /identity/v3/role_assignments?user.id=311403d2ab974eddb6f0bc18bcb7b8e0&scope.project.id=7ac0f0a66e994494baf51533717498da&role.id=1a4f694d53054a24b580138b1ba77f3c => generated 259 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:48:44.458040 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b708d7af-7062-4a08-9d43-ac82182c0e5f None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:48:44.459028 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b708d7af-7062-4a08-9d43-ac82182c0e5f None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:48:44.459509 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b708d7af-7062-4a08-9d43-ac82182c0e5f None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:48:44.461110 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 48/98] 10.222.0.27 () {54 vars in 927 bytes} [Fri Aug 5 19:48:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:48:44.466574 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7fcc46-bf3a-4035-9e79-d79897e4a302 None None] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:48:44.467450 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7fcc46-bf3a-4035-9e79-d79897e4a302 None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:48:44.467993 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7fcc46-bf3a-4035-9e79-d79897e4a302 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:48:44.491038 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.auth.core [None req-3c7fcc46-bf3a-4035-9e79-d79897e4a302 None None] MFA Rules not processed for user `03d1834a64cd4a43829827cb99a72747`. Rule list: `[]` (Enabled: `True`). {{(pid=80364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:48:44.492588 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-3c7fcc46-bf3a-4035-9e79-d79897e4a302 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:48:44.549649 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27e8841-2c82-4db5-8f31-3e4b651efa76 None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:48:44.550278 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27e8841-2c82-4db5-8f31-3e4b651efa76 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:48:44.550773 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27e8841-2c82-4db5-8f31-3e4b651efa76 None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:48:44.552325 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 49/99] 10.222.0.27 () {54 vars in 927 bytes} [Fri Aug 5 19:48:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 05 19:48:44.557454 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8d755a-e227-4480-89b0-ea1c02eebb42 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:48:44.557973 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8d755a-e227-4480-89b0-ea1c02eebb42 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:48:44.558484 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8d755a-e227-4480-89b0-ea1c02eebb42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:48:44.582898 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-6f8d755a-e227-4480-89b0-ea1c02eebb42 None None] MFA Rules not processed for user `03d1834a64cd4a43829827cb99a72747`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:48:44.587136 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-6f8d755a-e227-4480-89b0-ea1c02eebb42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:48:44.596860 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:48:44.614310 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 51/100] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Aug 5 19:48:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 05 19:48:44.675206 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7fcc46-bf3a-4035-9e79-d79897e4a302 None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:48:44.708749 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 50/101] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Aug 5 19:48:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 05 19:48:44.744276 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8d755a-e227-4480-89b0-ea1c02eebb42 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:48:44.825886 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7fcc46-bf3a-4035-9e79-d79897e4a302 None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:48:44.868154 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3f90a9-a8ca-4280-9eca-70c5e1f5ed08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:48:44.869266 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3f90a9-a8ca-4280-9eca-70c5e1f5ed08 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:48:44.869881 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3f90a9-a8ca-4280-9eca-70c5e1f5ed08 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:48:44.870363 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3f90a9-a8ca-4280-9eca-70c5e1f5ed08 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:48:44.872313 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd3f90a9-a8ca-4280-9eca-70c5e1f5ed08 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:48:44.874017 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd3f90a9-a8ca-4280-9eca-70c5e1f5ed08 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:48:44.876153 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 52/102] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Aug 5 19:48:44 2022] GET /identity/v3/domains/default => generated 199 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 19:48:44.883102 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3f90a9-a8ca-4280-9eca-70c5e1f5ed08 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:48:44.911881 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:48:44.917871 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8d755a-e227-4480-89b0-ea1c02eebb42 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:48:44.958886 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:48:44.982743 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c44d9f-1aa9-4e2f-86bc-fbce33f18245 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:48:44.983408 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c44d9f-1aa9-4e2f-86bc-fbce33f18245 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:48:44.983692 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c44d9f-1aa9-4e2f-86bc-fbce33f18245 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:48:44.983968 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c44d9f-1aa9-4e2f-86bc-fbce33f18245 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:48:45.006875 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c44d9f-1aa9-4e2f-86bc-fbce33f18245 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 19:48:45.013880 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-92c44d9f-1aa9-4e2f-86bc-fbce33f18245 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 05 19:48:45.013880 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:48:45.013880 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:48:45.013880 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:48:45.013880 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:48:45.013880 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:48:45.013880 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:48:45.013880 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:48:45.013880 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:48:45.013880 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:48:45.013880 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:48:45.013880 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:48:45.013880 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 19:48:45.013880 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 19:48:45.013880 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 19:48:45.013880 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 19:48:45.014798 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 19:48:45.014798 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 19:48:45.014798 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 19:48:45.014798 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 19:48:45.014798 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:48:45.014798 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:48:45.014798 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:48:45.014798 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:48:45.014798 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:48:45.014798 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:48:45.014798 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:48:45.014798 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Aug 05 19:48:45.014798 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:48:45.014798 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:48:45.014798 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:48:45.014798 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:48:45.015534 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:48:45.015534 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:48:45.015534 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:48:45.015534 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:48:45.015534 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 19:48:45.015534 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 19:48:45.015534 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 19:48:45.015534 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 19:48:45.015534 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 19:48:45.015534 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 19:48:45.015534 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 05 19:48:45.015534 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Aug 05 19:48:45.019414 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 51/103] 10.222.0.27 () {56 vars in 1099 bytes} [Fri Aug 5 19:48:44 2022] GET /identity/v3/roles/reader => generated 84 bytes in 276 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 05 19:48:45.030847 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c44d9f-1aa9-4e2f-86bc-fbce33f18245 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:48:45.038234 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3f90a9-a8ca-4280-9eca-70c5e1f5ed08 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:48:45.126834 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444d4a23-d64f-4ba7-8b83-3e8f314371a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:48:45.127822 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444d4a23-d64f-4ba7-8b83-3e8f314371a0 None admin] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:48:45.128381 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444d4a23-d64f-4ba7-8b83-3e8f314371a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:48:45.129015 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444d4a23-d64f-4ba7-8b83-3e8f314371a0 None admin] PATH_INFO: `/v3/projects` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:48:45.130606 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-444d4a23-d64f-4ba7-8b83-3e8f314371a0 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:48:45.134587 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-444d4a23-d64f-4ba7-8b83-3e8f314371a0 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:48:45.185374 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 53/104] 10.222.0.27 () {60 vars in 1141 bytes} [Fri Aug 5 19:48:44 2022] POST /identity/v3/projects => generated 302 bytes in 303 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 19:48:45.219405 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c44d9f-1aa9-4e2f-86bc-fbce33f18245 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:48:45.269807 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7cd7e3-136c-41f6-82e6-2312fe7a1f76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:48:45.270786 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7cd7e3-136c-41f6-82e6-2312fe7a1f76 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:48:45.271271 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7cd7e3-136c-41f6-82e6-2312fe7a1f76 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:48:45.271832 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7cd7e3-136c-41f6-82e6-2312fe7a1f76 None admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:48:45.272842 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b7cd7e3-136c-41f6-82e6-2312fe7a1f76 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:48:45.273448 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b7cd7e3-136c-41f6-82e6-2312fe7a1f76 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:48:45.275336 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b7cd7e3-136c-41f6-82e6-2312fe7a1f76 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:48:45.288492 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 52/105] 10.222.0.27 () {56 vars in 1101 bytes} [Fri Aug 5 19:48:45 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:48:45.295579 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444d4a23-d64f-4ba7-8b83-3e8f314371a0 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:48:45.445885 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444d4a23-d64f-4ba7-8b83-3e8f314371a0 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:48:45.460399 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:48:45.494915 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c25c1f7-58c0-4716-b2cb-718702799aec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:48:45.495526 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c25c1f7-58c0-4716-b2cb-718702799aec None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:48:45.495855 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c25c1f7-58c0-4716-b2cb-718702799aec None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:48:45.496093 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c25c1f7-58c0-4716-b2cb-718702799aec None admin] PATH_INFO: `/v3/users/f61db23dcfba4b319e8809ebabda24d7` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:48:45.515003 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c25c1f7-58c0-4716-b2cb-718702799aec None admin] RBAC: Authorizing `identity:get_user(user_id=f61db23dcfba4b319e8809ebabda24d7)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:48:45.517238 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c25c1f7-58c0-4716-b2cb-718702799aec None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:48:45.523833 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 54/106] 10.222.0.27 () {56 vars in 1177 bytes} [Fri Aug 5 19:48:45 2022] GET /identity/v3/users/f61db23dcfba4b319e8809ebabda24d7 => generated 289 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:48:45.525620 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7cd7e3-136c-41f6-82e6-2312fe7a1f76 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:48:45.569030 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:48:45.588543 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232e5429-cb3f-43cd-991e-5ec5f20079e1 None None] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:48:45.588949 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232e5429-cb3f-43cd-991e-5ec5f20079e1 None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:48:45.589173 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232e5429-cb3f-43cd-991e-5ec5f20079e1 None None] PATH_INFO: `/` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:48:45.590604 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 55/107] 10.222.0.27 () {54 vars in 927 bytes} [Fri Aug 5 19:48:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 05 19:48:45.596092 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3549f6-ac46-4708-9f8a-127f18ef6254 None None] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:48:45.596464 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3549f6-ac46-4708-9f8a-127f18ef6254 None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:48:45.596718 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3549f6-ac46-4708-9f8a-127f18ef6254 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:48:45.638727 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.auth.core [None req-6a3549f6-ac46-4708-9f8a-127f18ef6254 None None] MFA Rules not processed for user `03d1834a64cd4a43829827cb99a72747`. Rule list: `[]` (Enabled: `True`). {{(pid=80364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:48:45.640379 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-6a3549f6-ac46-4708-9f8a-127f18ef6254 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:48:45.689643 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7cd7e3-136c-41f6-82e6-2312fe7a1f76 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:48:45.729189 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d42f993-f83e-4fe2-afcd-f32301c5de2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:48:45.730249 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d42f993-f83e-4fe2-afcd-f32301c5de2f None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:48:45.730773 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d42f993-f83e-4fe2-afcd-f32301c5de2f None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:48:45.731387 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d42f993-f83e-4fe2-afcd-f32301c5de2f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:48:45.752629 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 56/108] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Aug 5 19:48:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Aug 05 19:48:45.768336 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b629e25-00b9-49fa-98ff-38d26fbe1b2b None None] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:48:45.768683 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b629e25-00b9-49fa-98ff-38d26fbe1b2b None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:48:45.768988 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b629e25-00b9-49fa-98ff-38d26fbe1b2b None None] PATH_INFO: `/` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:48:45.770502 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 57/109] 10.222.0.27 () {54 vars in 927 bytes} [Fri Aug 5 19:48:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 05 19:48:45.776565 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db299911-48db-4989-851b-b9f389be082b None None] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:48:45.777355 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db299911-48db-4989-851b-b9f389be082b None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:48:45.777912 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db299911-48db-4989-851b-b9f389be082b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:48:45.798546 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.auth.core [None req-db299911-48db-4989-851b-b9f389be082b None None] MFA Rules not processed for user `03d1834a64cd4a43829827cb99a72747`. Rule list: `[]` (Enabled: `True`). {{(pid=80364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:48:45.800316 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-db299911-48db-4989-851b-b9f389be082b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:48:45.872403 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d42f993-f83e-4fe2-afcd-f32301c5de2f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:48:45.873943 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d42f993-f83e-4fe2-afcd-f32301c5de2f None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:48:45.903521 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:48:45.907905 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 58/110] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Aug 5 19:48:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 05 19:48:45.912690 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db299911-48db-4989-851b-b9f389be082b None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:48:45.963535 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:48:46.063020 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 53/111] 10.222.0.27 () {58 vars in 1282 bytes} [Fri Aug 5 19:48:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 538 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 05 19:48:46.063722 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d42f993-f83e-4fe2-afcd-f32301c5de2f None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:48:46.159517 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db299911-48db-4989-851b-b9f389be082b None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:48:46.187032 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d42f993-f83e-4fe2-afcd-f32301c5de2f None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:48:46.211113 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ead618e-5e9f-4c51-ab3c-57fcf7da5813 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:48:46.211784 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ead618e-5e9f-4c51-ab3c-57fcf7da5813 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:48:46.212077 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ead618e-5e9f-4c51-ab3c-57fcf7da5813 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:48:46.212357 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ead618e-5e9f-4c51-ab3c-57fcf7da5813 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:48:46.215984 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:48:46.228943 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ead618e-5e9f-4c51-ab3c-57fcf7da5813 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 19:48:46.234081 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c16c83-f649-44a4-aac7-757f153fedd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:48:46.235054 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c16c83-f649-44a4-aac7-757f153fedd9 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:48:46.236051 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application [None req-8ead618e-5e9f-4c51-ab3c-57fcf7da5813 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 05 19:48:46.236051 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:48:46.236051 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:48:46.236051 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:48:46.236051 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:48:46.236051 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:48:46.236051 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:48:46.236051 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:48:46.236051 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:48:46.236051 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:48:46.236051 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:48:46.236051 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:48:46.236051 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 19:48:46.236051 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 19:48:46.236051 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 19:48:46.236051 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 19:48:46.236920 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 19:48:46.236920 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 19:48:46.236920 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 19:48:46.236920 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 19:48:46.236920 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:48:46.236920 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:48:46.236920 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:48:46.236920 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:48:46.236920 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:48:46.236920 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:48:46.236920 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:48:46.236920 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application with Lock( Aug 05 19:48:46.236920 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:48:46.236920 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:48:46.236920 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:48:46.236920 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:48:46.237697 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:48:46.237697 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:48:46.237697 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:48:46.237697 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:48:46.237697 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 19:48:46.237697 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 19:48:46.237697 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 19:48:46.237697 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 19:48:46.237697 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 19:48:46.237697 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 19:48:46.237697 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 05 19:48:46.237697 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Aug 05 19:48:46.237697 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 59/112] 10.222.0.27 () {56 vars in 1114 bytes} [Fri Aug 5 19:48:45 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 329 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:48:46.238849 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ead618e-5e9f-4c51-ab3c-57fcf7da5813 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:48:46.239398 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c16c83-f649-44a4-aac7-757f153fedd9 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:48:46.239874 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c16c83-f649-44a4-aac7-757f153fedd9 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:48:46.252584 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c16c83-f649-44a4-aac7-757f153fedd9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 19:48:46.258137 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-24c16c83-f649-44a4-aac7-757f153fedd9 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 05 19:48:46.258137 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:48:46.258137 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:48:46.258137 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:48:46.258137 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:48:46.258137 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:48:46.258137 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:48:46.258137 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:48:46.258137 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:48:46.258137 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:48:46.258137 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:48:46.258137 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:48:46.258137 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 19:48:46.258137 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 19:48:46.258137 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 19:48:46.258137 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 19:48:46.259005 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 19:48:46.259005 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 19:48:46.259005 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 19:48:46.259005 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 19:48:46.259005 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:48:46.259005 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:48:46.259005 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:48:46.259005 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:48:46.259005 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:48:46.259005 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:48:46.259005 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:48:46.259005 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Aug 05 19:48:46.259005 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:48:46.259005 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:48:46.259005 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:48:46.259005 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:48:46.259763 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:48:46.259763 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:48:46.259763 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:48:46.259763 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:48:46.259763 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 19:48:46.259763 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 19:48:46.259763 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 19:48:46.259763 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 19:48:46.259763 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 19:48:46.259763 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 19:48:46.259763 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 05 19:48:46.259763 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Aug 05 19:48:46.262179 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 54/113] 10.222.0.27 () {56 vars in 1099 bytes} [Fri Aug 5 19:48:46 2022] GET /identity/v3/roles/member => generated 84 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:48:46.263687 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2395f430-3729-4b3b-9832-30e0edfdefa3 None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:48:46.264185 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2395f430-3729-4b3b-9832-30e0edfdefa3 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:48:46.264729 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2395f430-3729-4b3b-9832-30e0edfdefa3 None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:48:46.266271 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 55/114] 10.222.0.27 () {54 vars in 927 bytes} [Fri Aug 5 19:48:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:48:46.267735 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2395f430-3729-4b3b-9832-30e0edfdefa3 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:48:46.384348 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:48:46.396859 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2395f430-3729-4b3b-9832-30e0edfdefa3 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:48:46.401861 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ead618e-5e9f-4c51-ab3c-57fcf7da5813 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:48:46.450926 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f630bd-723a-46c8-80d3-679633051115 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:48:46.451567 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f630bd-723a-46c8-80d3-679633051115 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:48:46.451850 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f630bd-723a-46c8-80d3-679633051115 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:48:46.452125 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f630bd-723a-46c8-80d3-679633051115 None admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:48:46.452959 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f630bd-723a-46c8-80d3-679633051115 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:48:46.453277 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f630bd-723a-46c8-80d3-679633051115 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:48:46.454797 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f630bd-723a-46c8-80d3-679633051115 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:48:46.455916 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8114fc9d-21d1-4737-a299-65eede3efada None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:48:46.456877 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8114fc9d-21d1-4737-a299-65eede3efada None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:48:46.457369 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8114fc9d-21d1-4737-a299-65eede3efada None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:48:46.457839 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8114fc9d-21d1-4737-a299-65eede3efada None admin] PATH_INFO: `/v3/projects/6f67d76798fb469c8ec9dd2d185572c3` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:48:46.459342 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8114fc9d-21d1-4737-a299-65eede3efada None admin] RBAC: Authorizing `identity:get_project(project_id=6f67d76798fb469c8ec9dd2d185572c3)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:48:46.461125 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8114fc9d-21d1-4737-a299-65eede3efada None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:48:46.463184 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 60/115] 10.222.0.27 () {56 vars in 1186 bytes} [Fri Aug 5 19:48:46 2022] GET /identity/v3/projects/6f67d76798fb469c8ec9dd2d185572c3 => generated 298 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:48:46.465454 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8114fc9d-21d1-4737-a299-65eede3efada None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:48:46.470781 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 56/116] 10.222.0.27 () {56 vars in 1111 bytes} [Fri Aug 5 19:48:46 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:48:46.471649 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43206a69-c990-4282-a85d-4c87829c8df7 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:48:46.471952 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43206a69-c990-4282-a85d-4c87829c8df7 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:48:46.472226 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43206a69-c990-4282-a85d-4c87829c8df7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:48:46.497161 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-43206a69-c990-4282-a85d-4c87829c8df7 None None] MFA Rules not processed for user `03d1834a64cd4a43829827cb99a72747`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:48:46.498383 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-43206a69-c990-4282-a85d-4c87829c8df7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:48:46.539073 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:48:46.575639 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 57/117] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Aug 5 19:48:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 19:48:46.576553 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43206a69-c990-4282-a85d-4c87829c8df7 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:48:46.590039 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8114fc9d-21d1-4737-a299-65eede3efada None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:48:46.629562 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a615e851-e87c-4f5b-8613-4b428ff9c99c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:48:46.630530 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a615e851-e87c-4f5b-8613-4b428ff9c99c None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:48:46.631017 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a615e851-e87c-4f5b-8613-4b428ff9c99c None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:48:46.631494 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a615e851-e87c-4f5b-8613-4b428ff9c99c None admin] PATH_INFO: `/v3/roles` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:48:46.632416 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a615e851-e87c-4f5b-8613-4b428ff9c99c None admin] RBAC: Adding query filter params (name=member) {{(pid=80364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:48:46.632967 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a615e851-e87c-4f5b-8613-4b428ff9c99c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:48:46.634638 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a615e851-e87c-4f5b-8613-4b428ff9c99c None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:48:46.645756 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 61/118] 10.222.0.27 () {56 vars in 1101 bytes} [Fri Aug 5 19:48:46 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:48:46.646926 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a615e851-e87c-4f5b-8613-4b428ff9c99c None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:48:46.672004 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:48:46.694184 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43206a69-c990-4282-a85d-4c87829c8df7 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:48:46.734403 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dea1b07-834f-4c5a-bd7a-cc1b0d423117 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:48:46.735043 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dea1b07-834f-4c5a-bd7a-cc1b0d423117 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:48:46.735879 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dea1b07-834f-4c5a-bd7a-cc1b0d423117 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:48:46.736214 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dea1b07-834f-4c5a-bd7a-cc1b0d423117 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:48:46.737068 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dea1b07-834f-4c5a-bd7a-cc1b0d423117 None admin] RBAC: Adding query filter params (role.id=b885e5d9af42483799785b59b26b6207, scope.project.id=6f67d76798fb469c8ec9dd2d185572c3, user.id=f61db23dcfba4b319e8809ebabda24d7) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:48:46.737430 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dea1b07-834f-4c5a-bd7a-cc1b0d423117 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:48:46.738917 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dea1b07-834f-4c5a-bd7a-cc1b0d423117 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:48:46.745230 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 58/119] 10.222.0.27 () {56 vars in 1374 bytes} [Fri Aug 5 19:48:46 2022] GET /identity/v3/role_assignments?user.id=f61db23dcfba4b319e8809ebabda24d7&scope.project.id=6f67d76798fb469c8ec9dd2d185572c3&role.id=b885e5d9af42483799785b59b26b6207 => generated 259 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:48:46.746080 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dea1b07-834f-4c5a-bd7a-cc1b0d423117 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:48:46.790592 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a615e851-e87c-4f5b-8613-4b428ff9c99c None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:48:46.806904 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:48:46.860770 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601db9a4-bea5-492e-8dab-5a763c9a70d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:48:46.862314 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601db9a4-bea5-492e-8dab-5a763c9a70d6 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:48:46.863062 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601db9a4-bea5-492e-8dab-5a763c9a70d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:48:46.863824 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601db9a4-bea5-492e-8dab-5a763c9a70d6 None admin] PATH_INFO: `/v3/users/311403d2ab974eddb6f0bc18bcb7b8e0` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:48:46.866414 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601db9a4-bea5-492e-8dab-5a763c9a70d6 None admin] RBAC: Authorizing `identity:get_user(user_id=311403d2ab974eddb6f0bc18bcb7b8e0)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:48:46.869388 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601db9a4-bea5-492e-8dab-5a763c9a70d6 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:48:46.873211 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 62/120] 10.222.0.27 () {56 vars in 1177 bytes} [Fri Aug 5 19:48:46 2022] GET /identity/v3/users/311403d2ab974eddb6f0bc18bcb7b8e0 => generated 275 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:48:46.874944 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601db9a4-bea5-492e-8dab-5a763c9a70d6 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:48:46.893458 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dea1b07-834f-4c5a-bd7a-cc1b0d423117 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:48:46.932553 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889e45db-ad54-415f-a12c-fcc640f4644e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:48:46.933315 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889e45db-ad54-415f-a12c-fcc640f4644e None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:48:46.933875 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889e45db-ad54-415f-a12c-fcc640f4644e None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:48:46.934153 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889e45db-ad54-415f-a12c-fcc640f4644e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:48:46.948114 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-889e45db-ad54-415f-a12c-fcc640f4644e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 19:48:46.955113 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-889e45db-ad54-415f-a12c-fcc640f4644e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 19:48:46.955113 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:48:46.955113 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:48:46.955113 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:48:46.955113 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:48:46.955113 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:48:46.955113 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:48:46.955113 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:48:46.955113 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:48:46.955113 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:48:46.955113 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:48:46.955113 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:48:46.955113 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 19:48:46.955113 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 19:48:46.955113 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 19:48:46.955113 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 19:48:46.958045 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 19:48:46.958045 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 19:48:46.958045 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 19:48:46.958045 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 19:48:46.958045 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:48:46.958045 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:48:46.958045 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:48:46.958045 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:48:46.958045 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:48:46.958045 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:48:46.958045 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:48:46.958045 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Aug 05 19:48:46.958045 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:48:46.958045 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:48:46.958045 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:48:46.958045 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:48:46.959488 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:48:46.959488 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:48:46.959488 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:48:46.959488 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:48:46.959488 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 19:48:46.959488 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 19:48:46.959488 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 19:48:46.959488 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 19:48:46.959488 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 19:48:46.959488 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 19:48:46.959488 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 19:48:46.959488 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Aug 05 19:48:46.959488 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 59/121] 10.222.0.27 () {56 vars in 1096 bytes} [Fri Aug 5 19:48:46 2022] GET /identity/v3/roles/admin => generated 83 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:48:46.959488 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b032c7c-c588-4d72-a703-9507ed70d326 None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:48:46.961799 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b032c7c-c588-4d72-a703-9507ed70d326 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:48:46.961799 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b032c7c-c588-4d72-a703-9507ed70d326 None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:48:46.961799 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 60/122] 10.222.0.27 () {54 vars in 927 bytes} [Fri Aug 5 19:48:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:48:46.961799 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b032c7c-c588-4d72-a703-9507ed70d326 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:48:47.065149 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601db9a4-bea5-492e-8dab-5a763c9a70d6 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:48:47.074792 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:48:47.080125 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:48:47.083937 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b032c7c-c588-4d72-a703-9507ed70d326 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:48:47.116260 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bfa7c1b-b2e9-4bfa-a4e2-ff14e28791e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:48:47.117849 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfa7c1b-b2e9-4bfa-a4e2-ff14e28791e9 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:48:47.118284 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfa7c1b-b2e9-4bfa-a4e2-ff14e28791e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:48:47.118741 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfa7c1b-b2e9-4bfa-a4e2-ff14e28791e9 None admin] PATH_INFO: `/v3/users/311403d2ab974eddb6f0bc18bcb7b8e0` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:48:47.120127 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bfa7c1b-b2e9-4bfa-a4e2-ff14e28791e9 None admin] RBAC: Authorizing `identity:get_user(user_id=311403d2ab974eddb6f0bc18bcb7b8e0)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:48:47.121799 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bfa7c1b-b2e9-4bfa-a4e2-ff14e28791e9 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:48:47.122030 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6961a7-2129-4405-a0dc-11a06a2a8993 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:48:47.122811 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6961a7-2129-4405-a0dc-11a06a2a8993 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:48:47.123227 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6961a7-2129-4405-a0dc-11a06a2a8993 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:48:47.123604 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6961a7-2129-4405-a0dc-11a06a2a8993 None admin] PATH_INFO: `/v3/projects/6f67d76798fb469c8ec9dd2d185572c3` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:48:47.124361 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 63/123] 10.222.0.27 () {56 vars in 1177 bytes} [Fri Aug 5 19:48:46 2022] GET /identity/v3/users/311403d2ab974eddb6f0bc18bcb7b8e0 => generated 275 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:48:47.125294 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bfa7c1b-b2e9-4bfa-a4e2-ff14e28791e9 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:48:47.129882 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec6961a7-2129-4405-a0dc-11a06a2a8993 None admin] RBAC: Authorizing `identity:get_project(project_id=6f67d76798fb469c8ec9dd2d185572c3)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:48:47.131296 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec6961a7-2129-4405-a0dc-11a06a2a8993 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:48:47.133245 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 61/124] 10.222.0.27 () {56 vars in 1186 bytes} [Fri Aug 5 19:48:46 2022] GET /identity/v3/projects/6f67d76798fb469c8ec9dd2d185572c3 => generated 298 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:48:47.140220 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ded9277-6e68-4315-8bfb-f70abea29ee8 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:48:47.140523 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ded9277-6e68-4315-8bfb-f70abea29ee8 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:48:47.140920 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ded9277-6e68-4315-8bfb-f70abea29ee8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:48:47.161506 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-8ded9277-6e68-4315-8bfb-f70abea29ee8 None None] MFA Rules not processed for user `03d1834a64cd4a43829827cb99a72747`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:48:47.162839 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-8ded9277-6e68-4315-8bfb-f70abea29ee8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:48:47.258101 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 62/125] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Aug 5 19:48:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 19:48:47.259009 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ded9277-6e68-4315-8bfb-f70abea29ee8 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:48:47.268934 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bfa7c1b-b2e9-4bfa-a4e2-ff14e28791e9 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:48:47.346422 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e9758d-360f-4312-8c09-853b52f950c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:48:47.347091 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e9758d-360f-4312-8c09-853b52f950c8 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:48:47.347417 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e9758d-360f-4312-8c09-853b52f950c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:48:47.347699 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e9758d-360f-4312-8c09-853b52f950c8 None admin] PATH_INFO: `/v3/roles` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:48:47.348555 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e9758d-360f-4312-8c09-853b52f950c8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:48:47.348877 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e9758d-360f-4312-8c09-853b52f950c8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:48:47.350299 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e9758d-360f-4312-8c09-853b52f950c8 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:48:47.372526 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 64/126] 10.222.0.27 () {56 vars in 1099 bytes} [Fri Aug 5 19:48:47 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:48:47.372526 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e9758d-360f-4312-8c09-853b52f950c8 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:48:47.372796 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:48:47.383129 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ded9277-6e68-4315-8bfb-f70abea29ee8 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:48:47.389649 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:48:47.423272 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0814cd5-3456-4769-a784-39254dd1f564 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:48:47.423966 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0814cd5-3456-4769-a784-39254dd1f564 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:48:47.424258 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0814cd5-3456-4769-a784-39254dd1f564 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:48:47.424689 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0814cd5-3456-4769-a784-39254dd1f564 None admin] PATH_INFO: `/v3/projects/6f67d76798fb469c8ec9dd2d185572c3` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:48:47.425921 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0814cd5-3456-4769-a784-39254dd1f564 None admin] RBAC: Authorizing `identity:get_project(project_id=6f67d76798fb469c8ec9dd2d185572c3)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:48:47.427463 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0814cd5-3456-4769-a784-39254dd1f564 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:48:47.429618 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 63/127] 10.222.0.27 () {56 vars in 1186 bytes} [Fri Aug 5 19:48:47 2022] GET /identity/v3/projects/6f67d76798fb469c8ec9dd2d185572c3 => generated 298 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:48:47.430519 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0814cd5-3456-4769-a784-39254dd1f564 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:48:47.501436 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e9758d-360f-4312-8c09-853b52f950c8 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:48:47.540028 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b24eabc-821c-43ea-af60-69a45b4ea679 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:48:47.540747 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b24eabc-821c-43ea-af60-69a45b4ea679 None admin] REQUEST_METHOD: `PUT` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:48:47.541029 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b24eabc-821c-43ea-af60-69a45b4ea679 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:48:47.541304 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b24eabc-821c-43ea-af60-69a45b4ea679 None admin] PATH_INFO: `/v3/projects/6f67d76798fb469c8ec9dd2d185572c3/users/311403d2ab974eddb6f0bc18bcb7b8e0/roles/dde0aa8028794132913bcba4d91858b8` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:48:47.543830 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b24eabc-821c-43ea-af60-69a45b4ea679 None admin] RBAC: Authorizing `identity:create_grant(project_id=6f67d76798fb469c8ec9dd2d185572c3, user_id=311403d2ab974eddb6f0bc18bcb7b8e0, role_id=dde0aa8028794132913bcba4d91858b8)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:48:47.545640 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b24eabc-821c-43ea-af60-69a45b4ea679 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:48:47.563235 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0814cd5-3456-4769-a784-39254dd1f564 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:48:47.568328 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 65/128] 10.222.0.27 () {58 vars in 1439 bytes} [Fri Aug 5 19:48:47 2022] PUT /identity/v3/projects/6f67d76798fb469c8ec9dd2d185572c3/users/311403d2ab974eddb6f0bc18bcb7b8e0/roles/dde0aa8028794132913bcba4d91858b8 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 19:48:47.569488 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b24eabc-821c-43ea-af60-69a45b4ea679 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:48:47.606581 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001ac7a6-1a92-4cbb-89c1-2ddae8934e47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:48:47.607682 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001ac7a6-1a92-4cbb-89c1-2ddae8934e47 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:48:47.608156 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001ac7a6-1a92-4cbb-89c1-2ddae8934e47 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:48:47.608668 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001ac7a6-1a92-4cbb-89c1-2ddae8934e47 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:48:47.626082 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-001ac7a6-1a92-4cbb-89c1-2ddae8934e47 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 19:48:47.631492 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-001ac7a6-1a92-4cbb-89c1-2ddae8934e47 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 05 19:48:47.631492 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:48:47.631492 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:48:47.631492 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:48:47.631492 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:48:47.631492 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:48:47.631492 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:48:47.631492 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:48:47.631492 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:48:47.631492 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:48:47.631492 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:48:47.631492 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:48:47.631492 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 19:48:47.631492 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 19:48:47.631492 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 19:48:47.631492 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 19:48:47.632381 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 19:48:47.632381 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 19:48:47.632381 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 19:48:47.632381 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 19:48:47.632381 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:48:47.632381 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:48:47.632381 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:48:47.632381 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:48:47.632381 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:48:47.632381 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:48:47.632381 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:48:47.632381 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Aug 05 19:48:47.632381 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:48:47.632381 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:48:47.632381 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:48:47.632381 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:48:47.633169 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:48:47.633169 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:48:47.633169 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:48:47.633169 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:48:47.633169 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 19:48:47.633169 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 19:48:47.633169 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 19:48:47.633169 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 19:48:47.633169 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 19:48:47.633169 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 19:48:47.633169 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 05 19:48:47.633169 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Aug 05 19:48:47.635641 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 64/129] 10.222.0.27 () {56 vars in 1099 bytes} [Fri Aug 5 19:48:47 2022] GET /identity/v3/roles/member => generated 84 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:48:47.636712 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001ac7a6-1a92-4cbb-89c1-2ddae8934e47 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:48:47.665258 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:48:47.695541 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:48:47.727607 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b24eabc-821c-43ea-af60-69a45b4ea679 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:48:47.769377 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001ac7a6-1a92-4cbb-89c1-2ddae8934e47 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:48:47.819137 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f41f85-bfbd-4087-8fe1-85bde1696725 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:48:47.824523 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f41f85-bfbd-4087-8fe1-85bde1696725 None admin] REQUEST_METHOD: `PUT` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:48:47.824523 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f41f85-bfbd-4087-8fe1-85bde1696725 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:48:47.824523 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f41f85-bfbd-4087-8fe1-85bde1696725 None admin] PATH_INFO: `/v3/projects/6f67d76798fb469c8ec9dd2d185572c3/users/311403d2ab974eddb6f0bc18bcb7b8e0/roles/1a4f694d53054a24b580138b1ba77f3c` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:48:47.824523 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f41f85-bfbd-4087-8fe1-85bde1696725 None admin] RBAC: Authorizing `identity:create_grant(project_id=6f67d76798fb469c8ec9dd2d185572c3, user_id=311403d2ab974eddb6f0bc18bcb7b8e0, role_id=1a4f694d53054a24b580138b1ba77f3c)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:48:47.824523 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f41f85-bfbd-4087-8fe1-85bde1696725 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:48:47.826777 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6be9b3a-33b0-4321-b0aa-d2b7665b0dbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:48:47.827681 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6be9b3a-33b0-4321-b0aa-d2b7665b0dbd None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:48:47.828186 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6be9b3a-33b0-4321-b0aa-d2b7665b0dbd None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:48:47.828759 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6be9b3a-33b0-4321-b0aa-d2b7665b0dbd None admin] PATH_INFO: `/v3/users/03d1834a64cd4a43829827cb99a72747` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:48:47.833900 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6be9b3a-33b0-4321-b0aa-d2b7665b0dbd None admin] RBAC: Authorizing `identity:get_user(user_id=03d1834a64cd4a43829827cb99a72747)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:48:47.836056 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6be9b3a-33b0-4321-b0aa-d2b7665b0dbd None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:48:47.838499 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 66/130] 10.222.0.27 () {56 vars in 1177 bytes} [Fri Aug 5 19:48:47 2022] GET /identity/v3/users/03d1834a64cd4a43829827cb99a72747 => generated 247 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:48:47.839552 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6be9b3a-33b0-4321-b0aa-d2b7665b0dbd None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:48:47.845285 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 65/131] 10.222.0.27 () {58 vars in 1439 bytes} [Fri Aug 5 19:48:47 2022] PUT /identity/v3/projects/6f67d76798fb469c8ec9dd2d185572c3/users/311403d2ab974eddb6f0bc18bcb7b8e0/roles/1a4f694d53054a24b580138b1ba77f3c => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 19:48:47.846030 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f41f85-bfbd-4087-8fe1-85bde1696725 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:48:47.959364 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:48:47.987502 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f41f85-bfbd-4087-8fe1-85bde1696725 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:48:48.036255 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715f8bb0-4f43-4e99-9eaa-72324758d8e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:48:48.036954 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715f8bb0-4f43-4e99-9eaa-72324758d8e4 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:48:48.037244 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715f8bb0-4f43-4e99-9eaa-72324758d8e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:48:48.037527 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715f8bb0-4f43-4e99-9eaa-72324758d8e4 None admin] PATH_INFO: `/v3/projects/6f67d76798fb469c8ec9dd2d185572c3` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:48:48.039255 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-715f8bb0-4f43-4e99-9eaa-72324758d8e4 None admin] RBAC: Authorizing `identity:get_project(project_id=6f67d76798fb469c8ec9dd2d185572c3)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:48:48.040830 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-715f8bb0-4f43-4e99-9eaa-72324758d8e4 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:48:48.042785 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 66/132] 10.222.0.27 () {56 vars in 1186 bytes} [Fri Aug 5 19:48:47 2022] GET /identity/v3/projects/6f67d76798fb469c8ec9dd2d185572c3 => generated 298 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:48:48.049973 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715f8bb0-4f43-4e99-9eaa-72324758d8e4 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:48:48.084911 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6be9b3a-33b0-4321-b0aa-d2b7665b0dbd None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:48:48.169442 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b391f9-a210-484c-b199-b246ef00cb9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:48:48.170283 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b391f9-a210-484c-b199-b246ef00cb9e None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:48:48.170591 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b391f9-a210-484c-b199-b246ef00cb9e None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:48:48.170939 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b391f9-a210-484c-b199-b246ef00cb9e None admin] PATH_INFO: `/v3/roles` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:48:48.171896 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b391f9-a210-484c-b199-b246ef00cb9e None admin] RBAC: Adding query filter params (name=member) {{(pid=80364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:48:48.172349 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b391f9-a210-484c-b199-b246ef00cb9e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:48:48.174224 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b391f9-a210-484c-b199-b246ef00cb9e None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:48:48.183453 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715f8bb0-4f43-4e99-9eaa-72324758d8e4 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:48:48.191641 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 67/133] 10.222.0.27 () {56 vars in 1101 bytes} [Fri Aug 5 19:48:47 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 352 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:48:48.197691 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b391f9-a210-484c-b199-b246ef00cb9e None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:48:48.222995 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:48:48.230238 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567422d5-e3d4-46e4-8738-7dbb79187896 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:48:48.231544 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567422d5-e3d4-46e4-8738-7dbb79187896 None admin] REQUEST_METHOD: `PUT` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:48:48.231840 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567422d5-e3d4-46e4-8738-7dbb79187896 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:48:48.232278 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567422d5-e3d4-46e4-8738-7dbb79187896 None admin] PATH_INFO: `/v3/projects/6f67d76798fb469c8ec9dd2d185572c3/users/03d1834a64cd4a43829827cb99a72747/roles/c173688a960b46e5be65f0ccd6fbd734` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:48:48.234687 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-567422d5-e3d4-46e4-8738-7dbb79187896 None admin] RBAC: Authorizing `identity:create_grant(project_id=6f67d76798fb469c8ec9dd2d185572c3, user_id=03d1834a64cd4a43829827cb99a72747, role_id=c173688a960b46e5be65f0ccd6fbd734)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:48:48.237554 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-567422d5-e3d4-46e4-8738-7dbb79187896 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:48:48.262377 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 67/134] 10.222.0.27 () {58 vars in 1439 bytes} [Fri Aug 5 19:48:48 2022] PUT /identity/v3/projects/6f67d76798fb469c8ec9dd2d185572c3/users/03d1834a64cd4a43829827cb99a72747/roles/c173688a960b46e5be65f0ccd6fbd734 => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 05 19:48:48.387808 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b391f9-a210-484c-b199-b246ef00cb9e None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:48:48.442435 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5071fa3-6a66-4332-a06b-a67b378b68e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:48:48.443493 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5071fa3-6a66-4332-a06b-a67b378b68e6 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:48:48.443990 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5071fa3-6a66-4332-a06b-a67b378b68e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:48:48.444459 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5071fa3-6a66-4332-a06b-a67b378b68e6 None admin] PATH_INFO: `/v3/users/311403d2ab974eddb6f0bc18bcb7b8e0` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:48:48.446042 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5071fa3-6a66-4332-a06b-a67b378b68e6 None admin] RBAC: Authorizing `identity:get_user(user_id=311403d2ab974eddb6f0bc18bcb7b8e0)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:48:48.447749 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5071fa3-6a66-4332-a06b-a67b378b68e6 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:48:48.450004 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 68/135] 10.222.0.27 () {56 vars in 1177 bytes} [Fri Aug 5 19:48:48 2022] GET /identity/v3/users/311403d2ab974eddb6f0bc18bcb7b8e0 => generated 275 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 19:48:48.456356 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567422d5-e3d4-46e4-8738-7dbb79187896 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:48:48.476690 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:48:48.587761 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567422d5-e3d4-46e4-8738-7dbb79187896 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:48:48.609861 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd075408-89c4-4d5e-b86e-fecd80fe7788 None None] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:48:48.610743 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd075408-89c4-4d5e-b86e-fecd80fe7788 None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:48:48.611124 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd075408-89c4-4d5e-b86e-fecd80fe7788 None None] PATH_INFO: `/` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:48:48.613419 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 69/136] 10.222.0.27 () {54 vars in 927 bytes} [Fri Aug 5 19:48:48 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:48:48.621543 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33797de8-3454-46c3-a078-d9d3747519ba None None] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:48:48.621846 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33797de8-3454-46c3-a078-d9d3747519ba None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:48:48.622135 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33797de8-3454-46c3-a078-d9d3747519ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:48:48.646941 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.auth.core [None req-33797de8-3454-46c3-a078-d9d3747519ba None None] MFA Rules not processed for user `03d1834a64cd4a43829827cb99a72747`. Rule list: `[]` (Enabled: `True`). {{(pid=80364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:48:48.648514 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-33797de8-3454-46c3-a078-d9d3747519ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:48:48.649352 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d108f54-1409-4213-9e7d-8b18d42c92ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:48:48.650494 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d108f54-1409-4213-9e7d-8b18d42c92ad None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:48:48.650964 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d108f54-1409-4213-9e7d-8b18d42c92ad None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:48:48.651425 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d108f54-1409-4213-9e7d-8b18d42c92ad None admin] PATH_INFO: `/v3/projects/7ac0f0a66e994494baf51533717498da` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:48:48.656523 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d108f54-1409-4213-9e7d-8b18d42c92ad None admin] RBAC: Authorizing `identity:get_project(project_id=7ac0f0a66e994494baf51533717498da)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:48:48.656523 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d108f54-1409-4213-9e7d-8b18d42c92ad None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:48:48.662635 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 68/137] 10.222.0.27 () {56 vars in 1186 bytes} [Fri Aug 5 19:48:48 2022] GET /identity/v3/projects/7ac0f0a66e994494baf51533717498da => generated 312 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:48:48.668766 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d108f54-1409-4213-9e7d-8b18d42c92ad None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:48:48.775763 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 70/138] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Aug 5 19:48:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 05 19:48:48.842164 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:48:48.845978 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33797de8-3454-46c3-a078-d9d3747519ba None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:48:48.858963 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:48:48.874791 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d108f54-1409-4213-9e7d-8b18d42c92ad None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:48:48.932051 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4fca54-ac4a-4fad-8c0b-ff361f37420e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:48:48.932772 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4fca54-ac4a-4fad-8c0b-ff361f37420e None admin] REQUEST_METHOD: `PUT` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:48:48.933058 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4fca54-ac4a-4fad-8c0b-ff361f37420e None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:48:48.933337 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4fca54-ac4a-4fad-8c0b-ff361f37420e None admin] PATH_INFO: `/v3/projects/7ac0f0a66e994494baf51533717498da/users/311403d2ab974eddb6f0bc18bcb7b8e0/roles/1a4f694d53054a24b580138b1ba77f3c` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:48:48.935353 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc4fca54-ac4a-4fad-8c0b-ff361f37420e None admin] RBAC: Authorizing `identity:create_grant(project_id=7ac0f0a66e994494baf51533717498da, user_id=311403d2ab974eddb6f0bc18bcb7b8e0, role_id=1a4f694d53054a24b580138b1ba77f3c)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:48:48.936986 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc4fca54-ac4a-4fad-8c0b-ff361f37420e None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:48:48.965882 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 69/139] 10.222.0.27 () {58 vars in 1439 bytes} [Fri Aug 5 19:48:48 2022] PUT /identity/v3/projects/7ac0f0a66e994494baf51533717498da/users/311403d2ab974eddb6f0bc18bcb7b8e0/roles/1a4f694d53054a24b580138b1ba77f3c => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 05 19:48:49.032251 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33797de8-3454-46c3-a078-d9d3747519ba None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:48:49.080590 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65affe11-4631-4c09-b4b1-c843da2701e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:48:49.081611 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65affe11-4631-4c09-b4b1-c843da2701e7 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:48:49.082104 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65affe11-4631-4c09-b4b1-c843da2701e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:48:49.082580 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65affe11-4631-4c09-b4b1-c843da2701e7 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:48:49.084078 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65affe11-4631-4c09-b4b1-c843da2701e7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:48:49.085835 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65affe11-4631-4c09-b4b1-c843da2701e7 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:48:49.088189 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 71/140] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Aug 5 19:48:48 2022] GET /identity/v3/domains/default => generated 199 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:48:49.093627 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4fca54-ac4a-4fad-8c0b-ff361f37420e None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:48:49.255581 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4fca54-ac4a-4fad-8c0b-ff361f37420e None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:48:49.298352 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f762c1-a32d-4b2f-864e-f6935f3f33a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:48:49.298923 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f762c1-a32d-4b2f-864e-f6935f3f33a5 None admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:48:49.299209 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f762c1-a32d-4b2f-864e-f6935f3f33a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:48:49.299488 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f762c1-a32d-4b2f-864e-f6935f3f33a5 None admin] PATH_INFO: `/v3/users` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:48:49.300211 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f762c1-a32d-4b2f-864e-f6935f3f33a5 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:48:49.301785 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f762c1-a32d-4b2f-864e-f6935f3f33a5 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:48:49.317375 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:48:49.336971 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 70/141] 10.222.0.27 () {60 vars in 1132 bytes} [Fri Aug 5 19:48:49 2022] POST /identity/v3/users => generated 283 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 19:48:49.713169 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13aa1148-7b43-43e0-9fa9-ef52ba1c0286 None None] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:48:49.713864 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13aa1148-7b43-43e0-9fa9-ef52ba1c0286 None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:48:49.714343 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13aa1148-7b43-43e0-9fa9-ef52ba1c0286 None None] PATH_INFO: `/` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:48:49.715920 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 72/142] 10.222.0.27 () {54 vars in 927 bytes} [Fri Aug 5 19:48:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:48:49.722004 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06dc7a51-a8da-410a-ba44-762b4951f484 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:48:49.722475 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06dc7a51-a8da-410a-ba44-762b4951f484 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:48:49.722559 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06dc7a51-a8da-410a-ba44-762b4951f484 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:48:49.741481 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-06dc7a51-a8da-410a-ba44-762b4951f484 None None] MFA Rules not processed for user `03d1834a64cd4a43829827cb99a72747`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:48:49.742754 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-06dc7a51-a8da-410a-ba44-762b4951f484 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:48:49.831182 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 71/143] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Aug 5 19:48:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 19:48:49.852482 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13aa1148-7b43-43e0-9fa9-ef52ba1c0286 None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:48:49.970987 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:48:50.005990 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13aa1148-7b43-43e0-9fa9-ef52ba1c0286 None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:48:50.051190 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a918eeb9-541a-478a-9c8d-29f2bcb17879 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:48:50.051844 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a918eeb9-541a-478a-9c8d-29f2bcb17879 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:48:50.052122 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a918eeb9-541a-478a-9c8d-29f2bcb17879 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:48:50.052396 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a918eeb9-541a-478a-9c8d-29f2bcb17879 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:48:50.065825 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a918eeb9-541a-478a-9c8d-29f2bcb17879 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 19:48:50.070754 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application [None req-a918eeb9-541a-478a-9c8d-29f2bcb17879 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 05 19:48:50.070754 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:48:50.070754 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:48:50.070754 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:48:50.070754 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:48:50.070754 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:48:50.070754 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:48:50.070754 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:48:50.070754 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:48:50.070754 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:48:50.070754 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:48:50.070754 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:48:50.070754 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 19:48:50.070754 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 19:48:50.070754 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 19:48:50.070754 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 19:48:50.071619 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 19:48:50.071619 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 19:48:50.071619 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 19:48:50.071619 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 19:48:50.071619 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:48:50.071619 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:48:50.071619 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:48:50.071619 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:48:50.071619 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:48:50.071619 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:48:50.071619 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:48:50.071619 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application with Lock( Aug 05 19:48:50.071619 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:48:50.071619 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:48:50.071619 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:48:50.071619 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:48:50.072378 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:48:50.072378 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:48:50.072378 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:48:50.072378 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:48:50.072378 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 19:48:50.072378 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 19:48:50.072378 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 19:48:50.072378 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 19:48:50.072378 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 19:48:50.072378 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 19:48:50.072378 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 05 19:48:50.072378 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Aug 05 19:48:50.072378 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 73/144] 10.222.0.27 () {56 vars in 1099 bytes} [Fri Aug 5 19:48:49 2022] GET /identity/v3/roles/reader => generated 84 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:48:50.079854 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06dc7a51-a8da-410a-ba44-762b4951f484 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:48:50.226945 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06dc7a51-a8da-410a-ba44-762b4951f484 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:48:50.255606 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:48:50.268829 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6eedfa-f296-4af2-8967-0a851ea76f87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:48:50.269958 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6eedfa-f296-4af2-8967-0a851ea76f87 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:48:50.270582 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6eedfa-f296-4af2-8967-0a851ea76f87 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:48:50.271054 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6eedfa-f296-4af2-8967-0a851ea76f87 None admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:48:50.271999 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf6eedfa-f296-4af2-8967-0a851ea76f87 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:48:50.272529 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf6eedfa-f296-4af2-8967-0a851ea76f87 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:48:50.274672 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf6eedfa-f296-4af2-8967-0a851ea76f87 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:48:50.284266 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 72/145] 10.222.0.27 () {56 vars in 1101 bytes} [Fri Aug 5 19:48:50 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:48:50.290033 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a918eeb9-541a-478a-9c8d-29f2bcb17879 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:48:50.398099 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a918eeb9-541a-478a-9c8d-29f2bcb17879 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:48:50.457312 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03cdd0b2-701f-4629-b224-79f1535635da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:48:50.458371 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cdd0b2-701f-4629-b224-79f1535635da None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:48:50.458854 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cdd0b2-701f-4629-b224-79f1535635da None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:48:50.459323 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cdd0b2-701f-4629-b224-79f1535635da None admin] PATH_INFO: `/v3/users/f61db23dcfba4b319e8809ebabda24d7` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:48:50.460837 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03cdd0b2-701f-4629-b224-79f1535635da None admin] RBAC: Authorizing `identity:get_user(user_id=f61db23dcfba4b319e8809ebabda24d7)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:48:50.462519 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03cdd0b2-701f-4629-b224-79f1535635da None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:48:50.464608 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 74/146] 10.222.0.27 () {56 vars in 1177 bytes} [Fri Aug 5 19:48:50 2022] GET /identity/v3/users/f61db23dcfba4b319e8809ebabda24d7 => generated 289 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:48:50.471457 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6eedfa-f296-4af2-8967-0a851ea76f87 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:48:50.606104 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6eedfa-f296-4af2-8967-0a851ea76f87 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:48:50.642157 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a884219e-0eb2-42ab-8a95-7e8634a0de62 None None] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:48:50.643057 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a884219e-0eb2-42ab-8a95-7e8634a0de62 None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:48:50.643761 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a884219e-0eb2-42ab-8a95-7e8634a0de62 None None] PATH_INFO: `/` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:48:50.646447 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 75/147] 10.222.0.27 () {54 vars in 927 bytes} [Fri Aug 5 19:48:50 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:48:50.652792 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50d391c-b66b-4391-b47a-bd1853a7c719 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:48:50.652792 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50d391c-b66b-4391-b47a-bd1853a7c719 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:48:50.652792 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50d391c-b66b-4391-b47a-bd1853a7c719 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:48:50.652792 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50d391c-b66b-4391-b47a-bd1853a7c719 None admin] PATH_INFO: `/v3/projects/6f67d76798fb469c8ec9dd2d185572c3` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:48:50.653316 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c50d391c-b66b-4391-b47a-bd1853a7c719 None admin] RBAC: Authorizing `identity:get_project(project_id=6f67d76798fb469c8ec9dd2d185572c3)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:48:50.654255 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c50d391c-b66b-4391-b47a-bd1853a7c719 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:48:50.656411 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 73/148] 10.222.0.27 () {56 vars in 1186 bytes} [Fri Aug 5 19:48:50 2022] GET /identity/v3/projects/6f67d76798fb469c8ec9dd2d185572c3 => generated 298 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:48:50.658588 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bf6679-f67a-4b82-bf93-620ce8cd061b None None] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:48:50.659320 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bf6679-f67a-4b82-bf93-620ce8cd061b None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:48:50.660062 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bf6679-f67a-4b82-bf93-620ce8cd061b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:48:50.662031 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50d391c-b66b-4391-b47a-bd1853a7c719 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:48:50.672154 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:48:50.706102 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.auth.core [None req-e4bf6679-f67a-4b82-bf93-620ce8cd061b None None] MFA Rules not processed for user `03d1834a64cd4a43829827cb99a72747`. Rule list: `[]` (Enabled: `True`). {{(pid=80364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:48:50.708423 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-e4bf6679-f67a-4b82-bf93-620ce8cd061b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:48:50.736456 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:48:50.833860 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50d391c-b66b-4391-b47a-bd1853a7c719 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:48:50.846279 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 76/149] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Aug 5 19:48:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 05 19:48:50.879982 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa92dd96-ef6c-4bc4-9e72-07394dfedbac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:48:50.880723 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa92dd96-ef6c-4bc4-9e72-07394dfedbac None admin] REQUEST_METHOD: `PUT` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:48:50.881015 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa92dd96-ef6c-4bc4-9e72-07394dfedbac None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:48:50.881296 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa92dd96-ef6c-4bc4-9e72-07394dfedbac None admin] PATH_INFO: `/v3/projects/6f67d76798fb469c8ec9dd2d185572c3/users/f61db23dcfba4b319e8809ebabda24d7/roles/b885e5d9af42483799785b59b26b6207` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:48:50.884055 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa92dd96-ef6c-4bc4-9e72-07394dfedbac None admin] RBAC: Authorizing `identity:create_grant(project_id=6f67d76798fb469c8ec9dd2d185572c3, user_id=f61db23dcfba4b319e8809ebabda24d7, role_id=b885e5d9af42483799785b59b26b6207)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:48:50.885570 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa92dd96-ef6c-4bc4-9e72-07394dfedbac None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:48:50.901000 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 74/150] 10.222.0.27 () {58 vars in 1439 bytes} [Fri Aug 5 19:48:50 2022] PUT /identity/v3/projects/6f67d76798fb469c8ec9dd2d185572c3/users/f61db23dcfba4b319e8809ebabda24d7/roles/b885e5d9af42483799785b59b26b6207 => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 19:48:50.928358 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4bf6679-f67a-4b82-bf93-620ce8cd061b None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:48:51.063547 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4bf6679-f67a-4b82-bf93-620ce8cd061b None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:48:51.098382 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:48:51.104068 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1857521-4a53-4dc6-848a-7542bbbe4f6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:48:51.104765 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1857521-4a53-4dc6-848a-7542bbbe4f6a None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:48:51.105049 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1857521-4a53-4dc6-848a-7542bbbe4f6a None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:48:51.105321 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1857521-4a53-4dc6-848a-7542bbbe4f6a None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:48:51.118298 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1857521-4a53-4dc6-848a-7542bbbe4f6a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 19:48:51.123347 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application [None req-b1857521-4a53-4dc6-848a-7542bbbe4f6a None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 05 19:48:51.123347 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:48:51.123347 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:48:51.123347 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:48:51.123347 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:48:51.123347 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:48:51.123347 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:48:51.123347 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:48:51.123347 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:48:51.123347 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:48:51.123347 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:48:51.123347 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:48:51.123347 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 19:48:51.123347 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 19:48:51.123347 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 19:48:51.123347 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 19:48:51.124211 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 19:48:51.124211 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 19:48:51.124211 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 19:48:51.124211 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 19:48:51.124211 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:48:51.124211 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:48:51.124211 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:48:51.124211 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:48:51.124211 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:48:51.124211 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:48:51.124211 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:48:51.124211 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application with Lock( Aug 05 19:48:51.124211 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:48:51.124211 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:48:51.124211 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:48:51.124211 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:48:51.125005 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:48:51.125005 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:48:51.125005 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:48:51.125005 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:48:51.125005 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 19:48:51.125005 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 19:48:51.125005 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 19:48:51.125005 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 19:48:51.125005 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 19:48:51.125005 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 19:48:51.125005 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 05 19:48:51.125005 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Aug 05 19:48:51.125005 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 77/151] 10.222.0.27 () {56 vars in 1114 bytes} [Fri Aug 5 19:48:50 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:48:51.132069 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa92dd96-ef6c-4bc4-9e72-07394dfedbac None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:48:51.269938 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa92dd96-ef6c-4bc4-9e72-07394dfedbac None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:48:51.291215 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:48:51.314466 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe84b59-2975-49d7-8d5c-f7d94f8bdca2 None None] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:48:51.314856 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe84b59-2975-49d7-8d5c-f7d94f8bdca2 None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:48:51.315275 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe84b59-2975-49d7-8d5c-f7d94f8bdca2 None None] PATH_INFO: `/` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:48:51.316939 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 78/152] 10.222.0.27 () {54 vars in 927 bytes} [Fri Aug 5 19:48:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:48:51.321943 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9dcf00-7304-4dbd-9983-a5e585706bd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:48:51.322596 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9dcf00-7304-4dbd-9983-a5e585706bd5 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:48:51.322905 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9dcf00-7304-4dbd-9983-a5e585706bd5 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:48:51.323165 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9dcf00-7304-4dbd-9983-a5e585706bd5 None admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:48:51.325916 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b9dcf00-7304-4dbd-9983-a5e585706bd5 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:48:51.326271 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b9dcf00-7304-4dbd-9983-a5e585706bd5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:48:51.328444 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b9dcf00-7304-4dbd-9983-a5e585706bd5 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:48:51.333097 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3144ca-0a33-42c6-ba88-bc15ffbe4915 None None] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:48:51.333380 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3144ca-0a33-42c6-ba88-bc15ffbe4915 None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:48:51.333670 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3144ca-0a33-42c6-ba88-bc15ffbe4915 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:48:51.346054 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 75/153] 10.222.0.27 () {56 vars in 1111 bytes} [Fri Aug 5 19:48:51 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:48:51.354825 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9dcf00-7304-4dbd-9983-a5e585706bd5 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:48:51.375312 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.auth.core [None req-ec3144ca-0a33-42c6-ba88-bc15ffbe4915 None None] MFA Rules not processed for user `03d1834a64cd4a43829827cb99a72747`. Rule list: `[]` (Enabled: `True`). {{(pid=80364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:48:51.376595 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-ec3144ca-0a33-42c6-ba88-bc15ffbe4915 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:48:51.527339 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9dcf00-7304-4dbd-9983-a5e585706bd5 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:48:51.582289 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f25318-c6e1-4eb2-b13b-8daf06070942 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:48:51.582965 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f25318-c6e1-4eb2-b13b-8daf06070942 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:48:51.583251 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f25318-c6e1-4eb2-b13b-8daf06070942 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:48:51.583794 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f25318-c6e1-4eb2-b13b-8daf06070942 None admin] PATH_INFO: `/v3/users/311403d2ab974eddb6f0bc18bcb7b8e0` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:48:51.586736 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f25318-c6e1-4eb2-b13b-8daf06070942 None admin] RBAC: Authorizing `identity:get_user(user_id=311403d2ab974eddb6f0bc18bcb7b8e0)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:48:51.588225 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f25318-c6e1-4eb2-b13b-8daf06070942 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:48:51.590505 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 76/154] 10.222.0.27 () {56 vars in 1177 bytes} [Fri Aug 5 19:48:51 2022] GET /identity/v3/users/311403d2ab974eddb6f0bc18bcb7b8e0 => generated 275 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 19:48:51.596166 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f25318-c6e1-4eb2-b13b-8daf06070942 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:48:51.674138 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 79/155] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Aug 5 19:48:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 342 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 19:48:51.689320 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:48:51.754250 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3144ca-0a33-42c6-ba88-bc15ffbe4915 None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:48:51.848123 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f25318-c6e1-4eb2-b13b-8daf06070942 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:48:51.972948 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7efa0c35-d92a-4fc1-a526-2e13fd89e0c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:48:51.973992 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efa0c35-d92a-4fc1-a526-2e13fd89e0c0 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:48:51.974485 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efa0c35-d92a-4fc1-a526-2e13fd89e0c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:48:51.980869 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efa0c35-d92a-4fc1-a526-2e13fd89e0c0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:48:52.103374 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3144ca-0a33-42c6-ba88-bc15ffbe4915 None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:48:52.179371 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:48:52.222841 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7efa0c35-d92a-4fc1-a526-2e13fd89e0c0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:48:52.229513 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7efa0c35-d92a-4fc1-a526-2e13fd89e0c0 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:48:52.233217 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792a323b-0393-43eb-9422-d58546ead7c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:48:52.233884 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792a323b-0393-43eb-9422-d58546ead7c7 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:48:52.234165 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792a323b-0393-43eb-9422-d58546ead7c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:48:52.234435 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792a323b-0393-43eb-9422-d58546ead7c7 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:48:52.261995 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-792a323b-0393-43eb-9422-d58546ead7c7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 19:48:52.269130 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application [None req-792a323b-0393-43eb-9422-d58546ead7c7 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 05 19:48:52.269130 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:48:52.269130 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:48:52.269130 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:48:52.269130 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:48:52.269130 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:48:52.269130 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:48:52.269130 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:48:52.269130 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:48:52.269130 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:48:52.269130 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:48:52.269130 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:48:52.269130 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 19:48:52.269130 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 19:48:52.269130 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 19:48:52.269130 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 19:48:52.270216 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 19:48:52.270216 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 19:48:52.270216 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 19:48:52.270216 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 19:48:52.270216 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:48:52.270216 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:48:52.270216 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:48:52.270216 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:48:52.270216 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:48:52.270216 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:48:52.270216 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:48:52.270216 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application with Lock( Aug 05 19:48:52.270216 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:48:52.270216 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:48:52.270216 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:48:52.270216 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:48:52.271055 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:48:52.271055 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:48:52.271055 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:48:52.271055 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:48:52.271055 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 19:48:52.271055 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 19:48:52.271055 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 19:48:52.271055 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 19:48:52.271055 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 19:48:52.271055 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 19:48:52.271055 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 05 19:48:52.271055 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Aug 05 19:48:52.271055 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 80/156] 10.222.0.27 () {56 vars in 1099 bytes} [Fri Aug 5 19:48:51 2022] GET /identity/v3/roles/member => generated 84 bytes in 517 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:48:52.271717 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d05ff83-5212-4401-8cd9-dc3c833c902b None None] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:48:52.271802 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d05ff83-5212-4401-8cd9-dc3c833c902b None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:48:52.272728 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d05ff83-5212-4401-8cd9-dc3c833c902b None None] PATH_INFO: `/` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:48:52.281863 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 81/157] 10.222.0.27 () {54 vars in 927 bytes} [Fri Aug 5 19:48:52 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:48:52.282788 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d05ff83-5212-4401-8cd9-dc3c833c902b None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:48:52.399056 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:48:52.464074 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d05ff83-5212-4401-8cd9-dc3c833c902b None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:48:52.501677 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4a28ed-2e8a-4912-a0d8-9eeaa69a489b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:48:52.503534 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4a28ed-2e8a-4912-a0d8-9eeaa69a489b None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:48:52.504043 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4a28ed-2e8a-4912-a0d8-9eeaa69a489b None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:48:52.504535 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4a28ed-2e8a-4912-a0d8-9eeaa69a489b None admin] PATH_INFO: `/v3/roles` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:48:52.505470 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4a28ed-2e8a-4912-a0d8-9eeaa69a489b None admin] RBAC: Adding query filter params (name=member) {{(pid=80364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:48:52.505981 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4a28ed-2e8a-4912-a0d8-9eeaa69a489b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:48:52.507557 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4a28ed-2e8a-4912-a0d8-9eeaa69a489b None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:48:52.517666 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 82/158] 10.222.0.27 () {56 vars in 1101 bytes} [Fri Aug 5 19:48:52 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:48:52.519289 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e28d35-7f55-46c0-9e19-6dcb69ba22be None None] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:48:52.520192 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e28d35-7f55-46c0-9e19-6dcb69ba22be None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:48:52.520699 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e28d35-7f55-46c0-9e19-6dcb69ba22be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:48:52.541163 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.auth.core [None req-d5e28d35-7f55-46c0-9e19-6dcb69ba22be None None] MFA Rules not processed for user `03d1834a64cd4a43829827cb99a72747`. Rule list: `[]` (Enabled: `True`). {{(pid=80364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:48:52.543253 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-d5e28d35-7f55-46c0-9e19-6dcb69ba22be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:48:52.599336 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:48:52.608528 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 77/159] 10.222.0.27 () {58 vars in 1282 bytes} [Fri Aug 5 19:48:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 1009 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 05 19:48:52.608528 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7efa0c35-d92a-4fc1-a526-2e13fd89e0c0 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:48:52.626344 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 83/160] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Aug 5 19:48:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 19:48:52.628739 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e28d35-7f55-46c0-9e19-6dcb69ba22be None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:48:52.752597 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7efa0c35-d92a-4fc1-a526-2e13fd89e0c0 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:48:52.801269 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd539ac-3f23-4331-83bb-595b5fa70c3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:48:52.802002 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd539ac-3f23-4331-83bb-595b5fa70c3d None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:48:52.802291 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd539ac-3f23-4331-83bb-595b5fa70c3d None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:48:52.802564 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd539ac-3f23-4331-83bb-595b5fa70c3d None admin] PATH_INFO: `/v3/users/311403d2ab974eddb6f0bc18bcb7b8e0` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:48:52.803823 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd539ac-3f23-4331-83bb-595b5fa70c3d None admin] RBAC: Authorizing `identity:get_user(user_id=311403d2ab974eddb6f0bc18bcb7b8e0)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:48:52.805305 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd539ac-3f23-4331-83bb-595b5fa70c3d None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:48:52.807115 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 78/161] 10.222.0.27 () {56 vars in 1177 bytes} [Fri Aug 5 19:48:52 2022] GET /identity/v3/users/311403d2ab974eddb6f0bc18bcb7b8e0 => generated 275 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:48:52.807776 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd539ac-3f23-4331-83bb-595b5fa70c3d None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:48:52.827611 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:48:52.862661 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e28d35-7f55-46c0-9e19-6dcb69ba22be None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:48:52.937304 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a697c3fd-27a7-4ee8-bd4f-c1c7680c8f33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:48:52.937899 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a697c3fd-27a7-4ee8-bd4f-c1c7680c8f33 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:48:52.938217 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a697c3fd-27a7-4ee8-bd4f-c1c7680c8f33 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:48:52.938496 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a697c3fd-27a7-4ee8-bd4f-c1c7680c8f33 None admin] PATH_INFO: `/v3/projects/6f67d76798fb469c8ec9dd2d185572c3` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:48:52.939714 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a697c3fd-27a7-4ee8-bd4f-c1c7680c8f33 None admin] RBAC: Authorizing `identity:get_project(project_id=6f67d76798fb469c8ec9dd2d185572c3)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:48:52.941218 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a697c3fd-27a7-4ee8-bd4f-c1c7680c8f33 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:48:52.942899 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 84/162] 10.222.0.27 () {56 vars in 1186 bytes} [Fri Aug 5 19:48:52 2022] GET /identity/v3/projects/6f67d76798fb469c8ec9dd2d185572c3 => generated 298 bytes in 315 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:48:52.943661 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a697c3fd-27a7-4ee8-bd4f-c1c7680c8f33 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:48:52.982623 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd539ac-3f23-4331-83bb-595b5fa70c3d None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:48:53.033968 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:48:53.054974 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89a3e8e-b050-4e73-b0b2-acb1048ca522 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:48:53.055688 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89a3e8e-b050-4e73-b0b2-acb1048ca522 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:48:53.055974 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89a3e8e-b050-4e73-b0b2-acb1048ca522 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:48:53.056252 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89a3e8e-b050-4e73-b0b2-acb1048ca522 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:48:53.077162 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c89a3e8e-b050-4e73-b0b2-acb1048ca522 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 19:48:53.082792 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-c89a3e8e-b050-4e73-b0b2-acb1048ca522 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 19:48:53.082792 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:48:53.082792 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:48:53.082792 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:48:53.082792 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:48:53.082792 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:48:53.082792 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:48:53.082792 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:48:53.082792 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:48:53.082792 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:48:53.082792 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:48:53.082792 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:48:53.082792 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 19:48:53.082792 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 19:48:53.082792 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 19:48:53.082792 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 19:48:53.083621 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 19:48:53.083621 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 19:48:53.083621 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 19:48:53.083621 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 19:48:53.083621 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:48:53.083621 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:48:53.083621 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:48:53.083621 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:48:53.083621 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:48:53.083621 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:48:53.083621 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:48:53.083621 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Aug 05 19:48:53.083621 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:48:53.083621 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:48:53.083621 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:48:53.083621 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:48:53.084404 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:48:53.084404 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:48:53.084404 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:48:53.084404 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:48:53.084404 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 19:48:53.084404 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 19:48:53.084404 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 19:48:53.084404 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 19:48:53.084404 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 19:48:53.084404 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 19:48:53.084404 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 19:48:53.084404 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Aug 05 19:48:53.084404 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 79/163] 10.222.0.27 () {56 vars in 1096 bytes} [Fri Aug 5 19:48:52 2022] GET /identity/v3/roles/admin => generated 83 bytes in 277 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:48:53.085077 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89a3e8e-b050-4e73-b0b2-acb1048ca522 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:48:53.145093 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a697c3fd-27a7-4ee8-bd4f-c1c7680c8f33 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:48:53.209788 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:48:53.221284 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a56dc45-be55-4dcc-a055-58c642d667c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:48:53.221928 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a56dc45-be55-4dcc-a055-58c642d667c1 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:48:53.222214 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a56dc45-be55-4dcc-a055-58c642d667c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:48:53.222491 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a56dc45-be55-4dcc-a055-58c642d667c1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:48:53.252631 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89a3e8e-b050-4e73-b0b2-acb1048ca522 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:48:53.291888 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470c88b7-f197-40c2-a344-023ffcac671f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:48:53.292528 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470c88b7-f197-40c2-a344-023ffcac671f None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:48:53.292820 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470c88b7-f197-40c2-a344-023ffcac671f None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:48:53.293092 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470c88b7-f197-40c2-a344-023ffcac671f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:48:53.293890 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470c88b7-f197-40c2-a344-023ffcac671f None admin] RBAC: Adding query filter params (role.id=dde0aa8028794132913bcba4d91858b8, scope.project.id=6f67d76798fb469c8ec9dd2d185572c3, user.id=311403d2ab974eddb6f0bc18bcb7b8e0) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:48:53.294196 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470c88b7-f197-40c2-a344-023ffcac671f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:48:53.295708 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470c88b7-f197-40c2-a344-023ffcac671f None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:48:53.302352 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 80/164] 10.222.0.27 () {56 vars in 1374 bytes} [Fri Aug 5 19:48:53 2022] GET /identity/v3/role_assignments?user.id=311403d2ab974eddb6f0bc18bcb7b8e0&scope.project.id=6f67d76798fb469c8ec9dd2d185572c3&role.id=dde0aa8028794132913bcba4d91858b8 => generated 608 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:48:53.303072 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470c88b7-f197-40c2-a344-023ffcac671f None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:48:53.353749 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a56dc45-be55-4dcc-a055-58c642d667c1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:48:53.355287 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a56dc45-be55-4dcc-a055-58c642d667c1 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:48:53.447771 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:48:53.481976 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470c88b7-f197-40c2-a344-023ffcac671f None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:48:53.545550 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 85/165] 10.222.0.27 () {58 vars in 1282 bytes} [Fri Aug 5 19:48:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 602 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 05 19:48:53.548377 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c765ecd8-8e13-486d-ac03-837813959d86 None None] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:48:53.553118 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c765ecd8-8e13-486d-ac03-837813959d86 None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:48:53.553640 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c765ecd8-8e13-486d-ac03-837813959d86 None None] PATH_INFO: `/` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:48:53.556333 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 86/166] 10.222.0.27 () {54 vars in 927 bytes} [Fri Aug 5 19:48:53 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:48:53.559369 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c765ecd8-8e13-486d-ac03-837813959d86 None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:48:53.563228 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc04295-0d72-4337-abc2-4911830c2f64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:48:53.564116 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc04295-0d72-4337-abc2-4911830c2f64 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:48:53.564623 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc04295-0d72-4337-abc2-4911830c2f64 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:48:53.565223 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc04295-0d72-4337-abc2-4911830c2f64 None admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:48:53.566167 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc04295-0d72-4337-abc2-4911830c2f64 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:48:53.566670 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc04295-0d72-4337-abc2-4911830c2f64 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:48:53.568267 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc04295-0d72-4337-abc2-4911830c2f64 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:48:53.580864 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:48:53.588143 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 81/167] 10.222.0.27 () {56 vars in 1099 bytes} [Fri Aug 5 19:48:53 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:48:53.589395 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660387f3-bf52-4d5d-bcf7-7ee20d6b78ba None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:48:53.589698 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660387f3-bf52-4d5d-bcf7-7ee20d6b78ba None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:48:53.589976 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660387f3-bf52-4d5d-bcf7-7ee20d6b78ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:48:53.619483 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-660387f3-bf52-4d5d-bcf7-7ee20d6b78ba None None] MFA Rules not processed for user `03d1834a64cd4a43829827cb99a72747`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:48:53.620716 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-660387f3-bf52-4d5d-bcf7-7ee20d6b78ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:48:53.745216 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c765ecd8-8e13-486d-ac03-837813959d86 None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:48:53.748098 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 82/168] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Aug 5 19:48:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 19:48:53.748904 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660387f3-bf52-4d5d-bcf7-7ee20d6b78ba None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:48:53.837228 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d603aa2-8a19-4d1e-8eee-68a6153f07e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:48:53.837880 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d603aa2-8a19-4d1e-8eee-68a6153f07e6 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:48:53.838202 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d603aa2-8a19-4d1e-8eee-68a6153f07e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:48:53.838476 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d603aa2-8a19-4d1e-8eee-68a6153f07e6 None admin] PATH_INFO: `/v3/projects/6f67d76798fb469c8ec9dd2d185572c3` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:48:53.839648 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d603aa2-8a19-4d1e-8eee-68a6153f07e6 None admin] RBAC: Authorizing `identity:get_project(project_id=6f67d76798fb469c8ec9dd2d185572c3)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:48:53.841166 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d603aa2-8a19-4d1e-8eee-68a6153f07e6 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:48:53.843276 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 87/169] 10.222.0.27 () {56 vars in 1186 bytes} [Fri Aug 5 19:48:53 2022] GET /identity/v3/projects/6f67d76798fb469c8ec9dd2d185572c3 => generated 298 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 19:48:53.844036 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d603aa2-8a19-4d1e-8eee-68a6153f07e6 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:48:53.868870 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:48:53.983609 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d603aa2-8a19-4d1e-8eee-68a6153f07e6 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:48:53.986363 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660387f3-bf52-4d5d-bcf7-7ee20d6b78ba None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:48:54.045218 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:48:54.050511 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075ec48b-563e-4b9c-915e-2d04d3add535 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:48:54.051164 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075ec48b-563e-4b9c-915e-2d04d3add535 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:48:54.051442 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075ec48b-563e-4b9c-915e-2d04d3add535 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:48:54.051715 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075ec48b-563e-4b9c-915e-2d04d3add535 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:48:54.055805 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35dac320-720d-48a6-bd93-1bbe65ff25f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:48:54.056815 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35dac320-720d-48a6-bd93-1bbe65ff25f8 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:48:54.057296 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35dac320-720d-48a6-bd93-1bbe65ff25f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:48:54.057779 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35dac320-720d-48a6-bd93-1bbe65ff25f8 None admin] PATH_INFO: `/v3/users/03d1834a64cd4a43829827cb99a72747` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:48:54.059152 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35dac320-720d-48a6-bd93-1bbe65ff25f8 None admin] RBAC: Authorizing `identity:get_user(user_id=03d1834a64cd4a43829827cb99a72747)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:48:54.060831 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35dac320-720d-48a6-bd93-1bbe65ff25f8 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:48:54.062665 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 83/170] 10.222.0.27 () {56 vars in 1177 bytes} [Fri Aug 5 19:48:53 2022] GET /identity/v3/users/03d1834a64cd4a43829827cb99a72747 => generated 247 bytes in 314 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:48:54.064160 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35dac320-720d-48a6-bd93-1bbe65ff25f8 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:48:54.077727 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-075ec48b-563e-4b9c-915e-2d04d3add535 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 19:48:54.083492 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application [None req-075ec48b-563e-4b9c-915e-2d04d3add535 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 05 19:48:54.083492 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:48:54.083492 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:48:54.083492 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:48:54.083492 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:48:54.083492 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:48:54.083492 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:48:54.083492 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:48:54.083492 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:48:54.083492 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:48:54.083492 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:48:54.083492 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:48:54.083492 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 19:48:54.083492 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 19:48:54.083492 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 19:48:54.083492 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 19:48:54.084374 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 19:48:54.084374 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 19:48:54.084374 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 19:48:54.084374 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 19:48:54.084374 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:48:54.084374 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:48:54.084374 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:48:54.084374 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:48:54.084374 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:48:54.084374 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:48:54.084374 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:48:54.084374 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application with Lock( Aug 05 19:48:54.084374 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:48:54.084374 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:48:54.084374 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:48:54.084374 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:48:54.085187 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:48:54.085187 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:48:54.085187 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:48:54.085187 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:48:54.085187 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 19:48:54.085187 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 19:48:54.085187 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 19:48:54.085187 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 19:48:54.085187 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 19:48:54.085187 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 19:48:54.085187 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 05 19:48:54.085187 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Aug 05 19:48:54.094174 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 88/171] 10.222.0.27 () {56 vars in 1099 bytes} [Fri Aug 5 19:48:53 2022] GET /identity/v3/roles/member => generated 84 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:48:54.101974 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bad4f0-95c9-47c8-aa60-40c946d69c1d None None] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:48:54.103454 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bad4f0-95c9-47c8-aa60-40c946d69c1d None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:48:54.104139 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bad4f0-95c9-47c8-aa60-40c946d69c1d None None] PATH_INFO: `/` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:48:54.107810 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 89/172] 10.222.0.27 () {54 vars in 927 bytes} [Fri Aug 5 19:48:54 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:48:54.108830 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bad4f0-95c9-47c8-aa60-40c946d69c1d None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:48:54.245026 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:48:54.250114 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35dac320-720d-48a6-bd93-1bbe65ff25f8 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:48:54.289087 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351fb54d-4ed1-4dbe-afef-d39a0d65fdbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:48:54.290067 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351fb54d-4ed1-4dbe-afef-d39a0d65fdbb None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:48:54.290548 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351fb54d-4ed1-4dbe-afef-d39a0d65fdbb None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:48:54.291041 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351fb54d-4ed1-4dbe-afef-d39a0d65fdbb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:48:54.292035 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-351fb54d-4ed1-4dbe-afef-d39a0d65fdbb None admin] RBAC: Adding query filter params (role.id=1a4f694d53054a24b580138b1ba77f3c, scope.project.id=6f67d76798fb469c8ec9dd2d185572c3, user.id=311403d2ab974eddb6f0bc18bcb7b8e0) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:48:54.292565 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-351fb54d-4ed1-4dbe-afef-d39a0d65fdbb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:48:54.294200 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-351fb54d-4ed1-4dbe-afef-d39a0d65fdbb None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:48:54.301521 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 84/173] 10.222.0.27 () {56 vars in 1374 bytes} [Fri Aug 5 19:48:54 2022] GET /identity/v3/role_assignments?user.id=311403d2ab974eddb6f0bc18bcb7b8e0&scope.project.id=6f67d76798fb469c8ec9dd2d185572c3&role.id=1a4f694d53054a24b580138b1ba77f3c => generated 608 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:48:54.302927 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c60a35-b152-4279-9372-7849b10057b7 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:48:54.303427 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c60a35-b152-4279-9372-7849b10057b7 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:48:54.303898 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c60a35-b152-4279-9372-7849b10057b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:48:54.312524 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bad4f0-95c9-47c8-aa60-40c946d69c1d None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:48:54.338617 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-42c60a35-b152-4279-9372-7849b10057b7 None None] MFA Rules not processed for user `03d1834a64cd4a43829827cb99a72747`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:48:54.340206 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-42c60a35-b152-4279-9372-7849b10057b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:48:54.359744 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce78f8f-1be7-472f-bea2-6c5b98d1a79b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:48:54.368657 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce78f8f-1be7-472f-bea2-6c5b98d1a79b None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:48:54.368657 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce78f8f-1be7-472f-bea2-6c5b98d1a79b None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:48:54.368657 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce78f8f-1be7-472f-bea2-6c5b98d1a79b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:48:54.475912 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:48:54.495712 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce78f8f-1be7-472f-bea2-6c5b98d1a79b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:48:54.497240 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce78f8f-1be7-472f-bea2-6c5b98d1a79b None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:48:54.503453 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 85/174] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Aug 5 19:48:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 19:48:54.504329 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c60a35-b152-4279-9372-7849b10057b7 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:48:54.635879 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c60a35-b152-4279-9372-7849b10057b7 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:48:54.639489 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 90/175] 10.222.0.27 () {58 vars in 1282 bytes} [Fri Aug 5 19:48:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 531 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 05 19:48:54.640011 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:48:54.646578 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf9d475-7a83-4069-a77f-415f95ab21ac None None] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:48:54.647265 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf9d475-7a83-4069-a77f-415f95ab21ac None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:48:54.647754 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf9d475-7a83-4069-a77f-415f95ab21ac None None] PATH_INFO: `/` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:48:54.650590 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 91/176] 10.222.0.27 () {54 vars in 927 bytes} [Fri Aug 5 19:48:54 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:48:54.651816 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf9d475-7a83-4069-a77f-415f95ab21ac None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:48:54.682561 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc603a2-0793-4cec-8c2c-fac370491611 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:48:54.683273 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc603a2-0793-4cec-8c2c-fac370491611 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:48:54.683600 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc603a2-0793-4cec-8c2c-fac370491611 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:48:54.683895 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc603a2-0793-4cec-8c2c-fac370491611 None admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:48:54.684816 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc603a2-0793-4cec-8c2c-fac370491611 None admin] RBAC: Adding query filter params (name=member) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:48:54.685126 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc603a2-0793-4cec-8c2c-fac370491611 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:48:54.686569 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc603a2-0793-4cec-8c2c-fac370491611 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:48:54.697975 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 86/177] 10.222.0.27 () {56 vars in 1101 bytes} [Fri Aug 5 19:48:54 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:48:54.698817 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc603a2-0793-4cec-8c2c-fac370491611 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:48:54.764880 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:48:54.795112 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf9d475-7a83-4069-a77f-415f95ab21ac None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:48:54.818560 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc603a2-0793-4cec-8c2c-fac370491611 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:48:54.841004 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176d6124-56d8-474e-904f-b97a28664306 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:48:54.841979 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176d6124-56d8-474e-904f-b97a28664306 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:48:54.842747 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176d6124-56d8-474e-904f-b97a28664306 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:48:54.843253 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176d6124-56d8-474e-904f-b97a28664306 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:48:54.859553 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e9a426-1771-435d-a613-6a3f4c83360c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:48:54.859963 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176d6124-56d8-474e-904f-b97a28664306 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 19:48:54.860259 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e9a426-1771-435d-a613-6a3f4c83360c None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:48:54.860565 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e9a426-1771-435d-a613-6a3f4c83360c None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:48:54.860843 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e9a426-1771-435d-a613-6a3f4c83360c None admin] PATH_INFO: `/v3/projects/6f67d76798fb469c8ec9dd2d185572c3` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:48:54.862015 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e9a426-1771-435d-a613-6a3f4c83360c None admin] RBAC: Authorizing `identity:get_project(project_id=6f67d76798fb469c8ec9dd2d185572c3)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:48:54.863452 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e9a426-1771-435d-a613-6a3f4c83360c None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:48:54.864893 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application [None req-176d6124-56d8-474e-904f-b97a28664306 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 19:48:54.864893 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:48:54.864893 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:48:54.864893 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:48:54.864893 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:48:54.864893 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:48:54.864893 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:48:54.864893 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:48:54.864893 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:48:54.864893 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:48:54.864893 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:48:54.864893 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:48:54.864893 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 19:48:54.864893 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 19:48:54.864893 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 19:48:54.864893 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 19:48:54.865737 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 19:48:54.865737 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 19:48:54.865737 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 19:48:54.865737 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 19:48:54.865737 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:48:54.865737 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:48:54.865737 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:48:54.865737 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:48:54.865737 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:48:54.865737 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:48:54.865737 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:48:54.865737 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application with Lock( Aug 05 19:48:54.865737 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:48:54.865737 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:48:54.865737 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:48:54.865737 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:48:54.866539 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:48:54.866539 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:48:54.866539 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:48:54.866539 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:48:54.866539 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 19:48:54.866539 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 19:48:54.866539 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 19:48:54.866539 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 19:48:54.866539 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 19:48:54.866539 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 19:48:54.866539 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 19:48:54.866539 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Aug 05 19:48:54.867135 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 87/178] 10.222.0.27 () {56 vars in 1186 bytes} [Fri Aug 5 19:48:54 2022] GET /identity/v3/projects/6f67d76798fb469c8ec9dd2d185572c3 => generated 298 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:48:54.867135 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fa936c-dbd4-41c1-9d00-7a39ce8d6277 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:48:54.868186 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 92/179] 10.222.0.27 () {56 vars in 1096 bytes} [Fri Aug 5 19:48:54 2022] GET /identity/v3/roles/admin => generated 83 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:48:54.868258 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fa936c-dbd4-41c1-9d00-7a39ce8d6277 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:48:54.868258 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fa936c-dbd4-41c1-9d00-7a39ce8d6277 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:48:54.868374 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176d6124-56d8-474e-904f-b97a28664306 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:48:54.893147 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-68fa936c-dbd4-41c1-9d00-7a39ce8d6277 None None] MFA Rules not processed for user `03d1834a64cd4a43829827cb99a72747`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:48:54.894683 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-68fa936c-dbd4-41c1-9d00-7a39ce8d6277 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:48:54.904158 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:48:54.965734 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 88/180] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Aug 5 19:48:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 19:48:54.970616 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc615ca0-72ec-44a8-acf6-02b3e48f8b62 None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:48:54.970947 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc615ca0-72ec-44a8-acf6-02b3e48f8b62 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:48:54.971236 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc615ca0-72ec-44a8-acf6-02b3e48f8b62 None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:48:54.973270 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 89/181] 10.222.0.27 () {54 vars in 927 bytes} [Fri Aug 5 19:48:54 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:48:54.973979 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc615ca0-72ec-44a8-acf6-02b3e48f8b62 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:48:54.981844 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176d6124-56d8-474e-904f-b97a28664306 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:48:55.021243 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddbcb6e2-f519-4bad-9faf-77d3900f8c3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:48:55.021887 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbcb6e2-f519-4bad-9faf-77d3900f8c3d None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:48:55.022167 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbcb6e2-f519-4bad-9faf-77d3900f8c3d None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:48:55.022438 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbcb6e2-f519-4bad-9faf-77d3900f8c3d None admin] PATH_INFO: `/v3/users/311403d2ab974eddb6f0bc18bcb7b8e0` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:48:55.023622 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddbcb6e2-f519-4bad-9faf-77d3900f8c3d None admin] RBAC: Authorizing `identity:get_user(user_id=311403d2ab974eddb6f0bc18bcb7b8e0)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:48:55.025357 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddbcb6e2-f519-4bad-9faf-77d3900f8c3d None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:48:55.027069 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 93/182] 10.222.0.27 () {56 vars in 1177 bytes} [Fri Aug 5 19:48:54 2022] GET /identity/v3/users/311403d2ab974eddb6f0bc18bcb7b8e0 => generated 275 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:48:55.032132 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddbcb6e2-f519-4bad-9faf-77d3900f8c3d None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:48:55.033560 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:48:55.084062 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc615ca0-72ec-44a8-acf6-02b3e48f8b62 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:48:55.116981 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ffcb9b-e339-481d-bec0-dda1f04258a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:48:55.117593 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ffcb9b-e339-481d-bec0-dda1f04258a8 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:48:55.117876 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ffcb9b-e339-481d-bec0-dda1f04258a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:48:55.118148 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ffcb9b-e339-481d-bec0-dda1f04258a8 None admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:48:55.118857 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ffcb9b-e339-481d-bec0-dda1f04258a8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:48:55.119158 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ffcb9b-e339-481d-bec0-dda1f04258a8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:48:55.120567 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ffcb9b-e339-481d-bec0-dda1f04258a8 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:48:55.128129 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:48:55.129204 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 90/183] 10.222.0.27 () {56 vars in 1099 bytes} [Fri Aug 5 19:48:54 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:48:55.130464 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0986e872-cfbe-4b59-b9dc-bf4b0d30923b None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:48:55.130811 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0986e872-cfbe-4b59-b9dc-bf4b0d30923b None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:48:55.131113 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0986e872-cfbe-4b59-b9dc-bf4b0d30923b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:48:55.136718 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddbcb6e2-f519-4bad-9faf-77d3900f8c3d None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:48:55.148348 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-0986e872-cfbe-4b59-b9dc-bf4b0d30923b None None] MFA Rules not processed for user `03d1834a64cd4a43829827cb99a72747`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:48:55.149492 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-0986e872-cfbe-4b59-b9dc-bf4b0d30923b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:48:55.169626 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f200080-6133-4593-9282-bf7c716fc126 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:48:55.170237 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f200080-6133-4593-9282-bf7c716fc126 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:48:55.170517 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f200080-6133-4593-9282-bf7c716fc126 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:48:55.170789 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f200080-6133-4593-9282-bf7c716fc126 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:48:55.171577 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f200080-6133-4593-9282-bf7c716fc126 None admin] RBAC: Adding query filter params (role.id=c173688a960b46e5be65f0ccd6fbd734, scope.project.id=6f67d76798fb469c8ec9dd2d185572c3, user.id=03d1834a64cd4a43829827cb99a72747) {{(pid=80364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:48:55.171882 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f200080-6133-4593-9282-bf7c716fc126 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:48:55.173844 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f200080-6133-4593-9282-bf7c716fc126 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:48:55.179576 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 94/184] 10.222.0.27 () {56 vars in 1374 bytes} [Fri Aug 5 19:48:55 2022] GET /identity/v3/role_assignments?user.id=03d1834a64cd4a43829827cb99a72747&scope.project.id=6f67d76798fb469c8ec9dd2d185572c3&role.id=c173688a960b46e5be65f0ccd6fbd734 => generated 608 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:48:55.180337 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f200080-6133-4593-9282-bf7c716fc126 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:48:55.229980 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 91/185] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Aug 5 19:48:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 19:48:55.234624 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297469ea-6d28-405e-b926-017e43afd305 None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:48:55.235413 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297469ea-6d28-405e-b926-017e43afd305 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:48:55.235822 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297469ea-6d28-405e-b926-017e43afd305 None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:48:55.242284 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 92/186] 10.222.0.27 () {54 vars in 927 bytes} [Fri Aug 5 19:48:55 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:48:55.242284 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297469ea-6d28-405e-b926-017e43afd305 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:48:55.302464 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:48:55.308484 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f200080-6133-4593-9282-bf7c716fc126 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:48:55.350086 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ab313b-649c-4d54-b857-d3e3f6f52f97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:48:55.350731 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ab313b-649c-4d54-b857-d3e3f6f52f97 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:48:55.351007 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ab313b-649c-4d54-b857-d3e3f6f52f97 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:48:55.351279 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ab313b-649c-4d54-b857-d3e3f6f52f97 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:48:55.353527 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ab313b-649c-4d54-b857-d3e3f6f52f97 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:48:55.355269 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ab313b-649c-4d54-b857-d3e3f6f52f97 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:48:55.359494 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 95/187] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Aug 5 19:48:55 2022] GET /identity/v3/domains/default => generated 199 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:48:55.359494 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ab313b-649c-4d54-b857-d3e3f6f52f97 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:48:55.380186 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297469ea-6d28-405e-b926-017e43afd305 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:48:55.394616 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:48:55.421091 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01839c83-bd14-472f-b408-79193f155085 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:48:55.421631 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01839c83-bd14-472f-b408-79193f155085 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:48:55.421957 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01839c83-bd14-472f-b408-79193f155085 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:48:55.422233 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01839c83-bd14-472f-b408-79193f155085 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:48:55.472201 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ab313b-649c-4d54-b857-d3e3f6f52f97 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:48:55.490934 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01839c83-bd14-472f-b408-79193f155085 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:48:55.492421 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01839c83-bd14-472f-b408-79193f155085 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:48:55.507906 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82912ce3-0a7c-4a2f-a385-958ebc9ba000 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:48:55.508698 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82912ce3-0a7c-4a2f-a385-958ebc9ba000 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:48:55.509074 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82912ce3-0a7c-4a2f-a385-958ebc9ba000 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:48:55.509349 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82912ce3-0a7c-4a2f-a385-958ebc9ba000 None admin] PATH_INFO: `/v3/users/aef034fd1a4f42618eb52ccce58d0014` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:48:55.523220 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82912ce3-0a7c-4a2f-a385-958ebc9ba000 None admin] RBAC: Authorizing `identity:get_user(user_id=aef034fd1a4f42618eb52ccce58d0014)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:48:55.524705 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82912ce3-0a7c-4a2f-a385-958ebc9ba000 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:48:55.527568 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 96/188] 10.222.0.27 () {56 vars in 1177 bytes} [Fri Aug 5 19:48:55 2022] GET /identity/v3/users/aef034fd1a4f42618eb52ccce58d0014 => generated 283 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:48:55.528746 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885b29a2-d621-4997-99f8-61f90b1d9711 None None] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:48:55.529015 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885b29a2-d621-4997-99f8-61f90b1d9711 None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:48:55.529288 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885b29a2-d621-4997-99f8-61f90b1d9711 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:48:55.540432 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:48:55.547101 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.auth.core [None req-885b29a2-d621-4997-99f8-61f90b1d9711 None None] MFA Rules not processed for user `03d1834a64cd4a43829827cb99a72747`. Rule list: `[]` (Enabled: `True`). {{(pid=80364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:48:55.548310 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-885b29a2-d621-4997-99f8-61f90b1d9711 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:48:55.601082 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 93/189] 10.222.0.27 () {58 vars in 1282 bytes} [Fri Aug 5 19:48:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 05 19:48:55.602125 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01839c83-bd14-472f-b408-79193f155085 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:48:55.621761 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 97/190] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Aug 5 19:48:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 19:48:55.622587 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885b29a2-d621-4997-99f8-61f90b1d9711 None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:48:55.641575 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:48:55.709060 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01839c83-bd14-472f-b408-79193f155085 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:48:55.722532 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885b29a2-d621-4997-99f8-61f90b1d9711 None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:48:55.743360 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26fb2f01-38c2-4f6b-b3d2-fbb5e3554e54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:48:55.743997 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fb2f01-38c2-4f6b-b3d2-fbb5e3554e54 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:48:55.744272 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fb2f01-38c2-4f6b-b3d2-fbb5e3554e54 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:48:55.744570 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fb2f01-38c2-4f6b-b3d2-fbb5e3554e54 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:48:55.755864 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26fb2f01-38c2-4f6b-b3d2-fbb5e3554e54 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 19:48:55.757945 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30da009-036e-4da3-9d7e-534f5af8cfc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:48:55.758555 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30da009-036e-4da3-9d7e-534f5af8cfc4 None admin] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:48:55.758829 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30da009-036e-4da3-9d7e-534f5af8cfc4 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:48:55.759096 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30da009-036e-4da3-9d7e-534f5af8cfc4 None admin] PATH_INFO: `/v3/users` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:48:55.759777 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30da009-036e-4da3-9d7e-534f5af8cfc4 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:48:55.760883 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-26fb2f01-38c2-4f6b-b3d2-fbb5e3554e54 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 05 19:48:55.760883 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:48:55.760883 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:48:55.760883 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:48:55.760883 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:48:55.760883 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:48:55.760883 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:48:55.760883 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:48:55.760883 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:48:55.760883 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:48:55.760883 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:48:55.760883 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:48:55.760883 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 19:48:55.760883 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 19:48:55.760883 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 19:48:55.760883 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 19:48:55.761761 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 19:48:55.761761 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 19:48:55.761761 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 19:48:55.761761 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 19:48:55.761761 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:48:55.761761 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:48:55.761761 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:48:55.761761 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:48:55.761761 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:48:55.761761 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:48:55.761761 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:48:55.761761 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Aug 05 19:48:55.761761 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:48:55.761761 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:48:55.761761 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:48:55.761761 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:48:55.762541 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:48:55.762541 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:48:55.762541 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:48:55.762541 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:48:55.762541 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 19:48:55.762541 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 19:48:55.762541 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 19:48:55.762541 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 19:48:55.762541 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 19:48:55.762541 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 19:48:55.762541 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 05 19:48:55.762541 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Aug 05 19:48:55.763129 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30da009-036e-4da3-9d7e-534f5af8cfc4 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:48:55.763203 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 94/191] 10.222.0.27 () {56 vars in 1099 bytes} [Fri Aug 5 19:48:55 2022] GET /identity/v3/roles/reader => generated 84 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:48:55.765357 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26fb2f01-38c2-4f6b-b3d2-fbb5e3554e54 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:48:55.769240 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:48:55.784572 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 98/192] 10.222.0.27 () {60 vars in 1132 bytes} [Fri Aug 5 19:48:55 2022] POST /identity/v3/users => generated 297 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 19:48:55.785495 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30da009-036e-4da3-9d7e-534f5af8cfc4 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:48:55.879279 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:48:55.889289 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26fb2f01-38c2-4f6b-b3d2-fbb5e3554e54 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:48:55.911242 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30da009-036e-4da3-9d7e-534f5af8cfc4 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:48:55.933765 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b401bd90-7900-4fb0-8284-df415a5d97a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:48:55.934403 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b401bd90-7900-4fb0-8284-df415a5d97a4 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:48:55.934715 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b401bd90-7900-4fb0-8284-df415a5d97a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:48:55.934993 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b401bd90-7900-4fb0-8284-df415a5d97a4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:48:55.951999 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fba0c03-ff23-4cd4-8954-7a3016e74b77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:48:55.952467 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fba0c03-ff23-4cd4-8954-7a3016e74b77 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:48:55.952699 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fba0c03-ff23-4cd4-8954-7a3016e74b77 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:48:55.952699 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fba0c03-ff23-4cd4-8954-7a3016e74b77 None admin] PATH_INFO: `/v3/projects/7ac0f0a66e994494baf51533717498da` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:48:55.955194 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fba0c03-ff23-4cd4-8954-7a3016e74b77 None admin] RBAC: Authorizing `identity:get_project(project_id=7ac0f0a66e994494baf51533717498da)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:48:55.956689 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fba0c03-ff23-4cd4-8954-7a3016e74b77 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:48:55.963269 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 99/193] 10.222.0.27 () {56 vars in 1186 bytes} [Fri Aug 5 19:48:55 2022] GET /identity/v3/projects/7ac0f0a66e994494baf51533717498da => generated 312 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:48:55.963269 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fba0c03-ff23-4cd4-8954-7a3016e74b77 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:48:56.037355 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b401bd90-7900-4fb0-8284-df415a5d97a4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:48:56.038858 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b401bd90-7900-4fb0-8284-df415a5d97a4 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:48:56.042017 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:48:56.099709 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fba0c03-ff23-4cd4-8954-7a3016e74b77 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:48:56.137819 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1583a8-32a7-4066-a795-51667d062ad2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:48:56.138480 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1583a8-32a7-4066-a795-51667d062ad2 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:48:56.138758 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1583a8-32a7-4066-a795-51667d062ad2 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:48:56.139130 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1583a8-32a7-4066-a795-51667d062ad2 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:48:56.150759 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1583a8-32a7-4066-a795-51667d062ad2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 19:48:56.156664 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application [None req-3d1583a8-32a7-4066-a795-51667d062ad2 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 05 19:48:56.156664 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:48:56.156664 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:48:56.156664 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:48:56.156664 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:48:56.156664 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:48:56.156664 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:48:56.156664 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:48:56.156664 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:48:56.156664 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:48:56.156664 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:48:56.156664 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:48:56.156664 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 19:48:56.156664 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 19:48:56.156664 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 19:48:56.156664 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 19:48:56.157640 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 19:48:56.157640 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 19:48:56.157640 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 19:48:56.157640 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 19:48:56.157640 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:48:56.157640 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:48:56.157640 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:48:56.157640 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:48:56.157640 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:48:56.157640 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:48:56.157640 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:48:56.157640 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application with Lock( Aug 05 19:48:56.157640 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:48:56.157640 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:48:56.157640 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:48:56.157640 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:48:56.158420 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:48:56.158420 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:48:56.158420 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:48:56.158420 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:48:56.158420 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 19:48:56.158420 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 19:48:56.158420 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 19:48:56.158420 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 19:48:56.158420 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 19:48:56.158420 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 19:48:56.158420 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 05 19:48:56.158420 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Aug 05 19:48:56.158420 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 100/194] 10.222.0.27 () {56 vars in 1114 bytes} [Fri Aug 5 19:48:55 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:48:56.159125 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1583a8-32a7-4066-a795-51667d062ad2 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:48:56.167848 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:48:56.173505 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 95/195] 10.222.0.27 () {58 vars in 1282 bytes} [Fri Aug 5 19:48:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 411 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 05 19:48:56.174254 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b401bd90-7900-4fb0-8284-df415a5d97a4 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:48:56.283636 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1583a8-32a7-4066-a795-51667d062ad2 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:48:56.301163 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b401bd90-7900-4fb0-8284-df415a5d97a4 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:48:56.302891 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:48:56.333066 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b0f045-62f5-4d35-ab6b-ea5f1eacf64f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:48:56.334904 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b0f045-62f5-4d35-ab6b-ea5f1eacf64f None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:48:56.335481 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b0f045-62f5-4d35-ab6b-ea5f1eacf64f None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:48:56.336050 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b0f045-62f5-4d35-ab6b-ea5f1eacf64f None admin] PATH_INFO: `/v3/roles` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:48:56.337065 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b0f045-62f5-4d35-ab6b-ea5f1eacf64f None admin] RBAC: Adding query filter params (name=reader) {{(pid=80364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:48:56.337545 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b0f045-62f5-4d35-ab6b-ea5f1eacf64f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:48:56.339321 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b0f045-62f5-4d35-ab6b-ea5f1eacf64f None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:48:56.339719 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5880faac-f38c-4688-9624-67f86142264b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:48:56.340410 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5880faac-f38c-4688-9624-67f86142264b None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:48:56.340815 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5880faac-f38c-4688-9624-67f86142264b None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:48:56.341161 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5880faac-f38c-4688-9624-67f86142264b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:48:56.342031 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5880faac-f38c-4688-9624-67f86142264b None admin] RBAC: Adding query filter params (role.id=1a4f694d53054a24b580138b1ba77f3c, scope.project.id=7ac0f0a66e994494baf51533717498da, user.id=311403d2ab974eddb6f0bc18bcb7b8e0) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:48:56.342417 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5880faac-f38c-4688-9624-67f86142264b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:48:56.343929 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5880faac-f38c-4688-9624-67f86142264b None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:48:56.350710 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 96/196] 10.222.0.27 () {56 vars in 1374 bytes} [Fri Aug 5 19:48:56 2022] GET /identity/v3/role_assignments?user.id=311403d2ab974eddb6f0bc18bcb7b8e0&scope.project.id=7ac0f0a66e994494baf51533717498da&role.id=1a4f694d53054a24b580138b1ba77f3c => generated 608 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:48:56.356773 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5880faac-f38c-4688-9624-67f86142264b None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:48:56.366165 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 101/197] 10.222.0.27 () {56 vars in 1101 bytes} [Fri Aug 5 19:48:56 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:48:56.366165 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b0f045-62f5-4d35-ab6b-ea5f1eacf64f None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:48:56.502553 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:48:56.549444 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5880faac-f38c-4688-9624-67f86142264b None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:48:56.566759 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b0f045-62f5-4d35-ab6b-ea5f1eacf64f None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:48:56.584743 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63aaba5f-e025-4e12-98a0-128a0b1a5b78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:48:56.585375 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63aaba5f-e025-4e12-98a0-128a0b1a5b78 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:48:56.585813 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63aaba5f-e025-4e12-98a0-128a0b1a5b78 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:48:56.585917 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63aaba5f-e025-4e12-98a0-128a0b1a5b78 None admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:48:56.586689 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63aaba5f-e025-4e12-98a0-128a0b1a5b78 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:48:56.586995 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63aaba5f-e025-4e12-98a0-128a0b1a5b78 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:48:56.588427 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63aaba5f-e025-4e12-98a0-128a0b1a5b78 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:48:56.597353 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 97/198] 10.222.0.27 () {56 vars in 1111 bytes} [Fri Aug 5 19:48:56 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:48:56.597981 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63aaba5f-e025-4e12-98a0-128a0b1a5b78 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:48:56.613014 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b834fd-c34b-4082-ae5c-2ce85576f67b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:48:56.614015 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b834fd-c34b-4082-ae5c-2ce85576f67b None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:48:56.614510 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b834fd-c34b-4082-ae5c-2ce85576f67b None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:48:56.615055 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b834fd-c34b-4082-ae5c-2ce85576f67b None admin] PATH_INFO: `/v3/projects/6440bf75a9134de7ba2c5fb9c23955e2` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:48:56.616525 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b834fd-c34b-4082-ae5c-2ce85576f67b None admin] RBAC: Authorizing `identity:get_project(project_id=6440bf75a9134de7ba2c5fb9c23955e2)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:48:56.618255 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b834fd-c34b-4082-ae5c-2ce85576f67b None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:48:56.620572 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 102/199] 10.222.0.27 () {56 vars in 1186 bytes} [Fri Aug 5 19:48:56 2022] GET /identity/v3/projects/6440bf75a9134de7ba2c5fb9c23955e2 => generated 302 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:48:56.621601 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b834fd-c34b-4082-ae5c-2ce85576f67b None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:48:56.628999 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:48:56.714742 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63aaba5f-e025-4e12-98a0-128a0b1a5b78 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:48:56.748688 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b834fd-c34b-4082-ae5c-2ce85576f67b None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:48:56.765293 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d68d877-129b-4fb3-8c4b-d40a498b50ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:48:56.766741 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d68d877-129b-4fb3-8c4b-d40a498b50ea None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:48:56.767421 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d68d877-129b-4fb3-8c4b-d40a498b50ea None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:48:56.767703 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d68d877-129b-4fb3-8c4b-d40a498b50ea None admin] PATH_INFO: `/v3/users/f61db23dcfba4b319e8809ebabda24d7` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:48:56.769019 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d68d877-129b-4fb3-8c4b-d40a498b50ea None admin] RBAC: Authorizing `identity:get_user(user_id=f61db23dcfba4b319e8809ebabda24d7)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:48:56.771049 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d68d877-129b-4fb3-8c4b-d40a498b50ea None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:48:56.774313 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 98/200] 10.222.0.27 () {56 vars in 1177 bytes} [Fri Aug 5 19:48:56 2022] GET /identity/v3/users/f61db23dcfba4b319e8809ebabda24d7 => generated 289 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:48:56.775658 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d68d877-129b-4fb3-8c4b-d40a498b50ea None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:48:56.793893 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:48:56.801903 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f6ff5c-db56-4abd-a9a2-f78101a7c053 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:48:56.802900 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f6ff5c-db56-4abd-a9a2-f78101a7c053 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:48:56.803802 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f6ff5c-db56-4abd-a9a2-f78101a7c053 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:48:56.803979 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f6ff5c-db56-4abd-a9a2-f78101a7c053 None admin] PATH_INFO: `/v3/users/aef034fd1a4f42618eb52ccce58d0014` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:48:56.805327 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f6ff5c-db56-4abd-a9a2-f78101a7c053 None admin] RBAC: Authorizing `identity:get_user(user_id=aef034fd1a4f42618eb52ccce58d0014)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:48:56.806917 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f6ff5c-db56-4abd-a9a2-f78101a7c053 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:48:56.811203 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 103/201] 10.222.0.27 () {56 vars in 1177 bytes} [Fri Aug 5 19:48:56 2022] GET /identity/v3/users/aef034fd1a4f42618eb52ccce58d0014 => generated 283 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:48:56.811203 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f6ff5c-db56-4abd-a9a2-f78101a7c053 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:48:56.923559 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:48:56.931425 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d68d877-129b-4fb3-8c4b-d40a498b50ea None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:48:56.932875 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f6ff5c-db56-4abd-a9a2-f78101a7c053 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:48:56.969863 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0220285-7ddd-4783-9211-7702abefc4a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:48:56.970863 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0220285-7ddd-4783-9211-7702abefc4a4 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:48:56.971370 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0220285-7ddd-4783-9211-7702abefc4a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:48:56.971844 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0220285-7ddd-4783-9211-7702abefc4a4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:48:56.976624 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0288d51-1053-44fa-8a5d-f96eb62dc1c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:48:56.977244 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0288d51-1053-44fa-8a5d-f96eb62dc1c3 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:48:56.977525 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0288d51-1053-44fa-8a5d-f96eb62dc1c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:48:56.977802 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0288d51-1053-44fa-8a5d-f96eb62dc1c3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:48:56.978606 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0288d51-1053-44fa-8a5d-f96eb62dc1c3 None admin] RBAC: Adding query filter params (role.id=c173688a960b46e5be65f0ccd6fbd734, scope.project.id=6440bf75a9134de7ba2c5fb9c23955e2, user.id=aef034fd1a4f42618eb52ccce58d0014) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:48:56.978919 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0288d51-1053-44fa-8a5d-f96eb62dc1c3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:48:56.980428 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0288d51-1053-44fa-8a5d-f96eb62dc1c3 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:48:56.987775 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 99/202] 10.222.0.27 () {56 vars in 1374 bytes} [Fri Aug 5 19:48:56 2022] GET /identity/v3/role_assignments?user.id=aef034fd1a4f42618eb52ccce58d0014&scope.project.id=6440bf75a9134de7ba2c5fb9c23955e2&role.id=c173688a960b46e5be65f0ccd6fbd734 => generated 259 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:48:56.988876 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0288d51-1053-44fa-8a5d-f96eb62dc1c3 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:48:57.125273 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:48:57.130025 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0220285-7ddd-4783-9211-7702abefc4a4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:48:57.133665 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0220285-7ddd-4783-9211-7702abefc4a4 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:48:57.242097 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0288d51-1053-44fa-8a5d-f96eb62dc1c3 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:48:57.354996 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408db24a-7626-4f07-80c7-0910cd42453b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:48:57.356411 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408db24a-7626-4f07-80c7-0910cd42453b None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:48:57.357143 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408db24a-7626-4f07-80c7-0910cd42453b None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:48:57.357823 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408db24a-7626-4f07-80c7-0910cd42453b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:48:57.403218 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 104/203] 10.222.0.27 () {58 vars in 1282 bytes} [Fri Aug 5 19:48:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 594 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 05 19:48:57.409743 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0220285-7ddd-4783-9211-7702abefc4a4 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:48:57.448127 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:48:57.563887 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-408db24a-7626-4f07-80c7-0910cd42453b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:48:57.566291 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-408db24a-7626-4f07-80c7-0910cd42453b None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:48:57.660286 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:48:57.667834 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0220285-7ddd-4783-9211-7702abefc4a4 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:48:57.730292 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5678d7-05a3-489d-b8db-a53714521653 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:48:57.731031 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5678d7-05a3-489d-b8db-a53714521653 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:48:57.731308 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5678d7-05a3-489d-b8db-a53714521653 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:48:57.731577 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5678d7-05a3-489d-b8db-a53714521653 None admin] PATH_INFO: `/v3/projects/6f67d76798fb469c8ec9dd2d185572c3` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:48:57.732844 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff5678d7-05a3-489d-b8db-a53714521653 None admin] RBAC: Authorizing `identity:get_project(project_id=6f67d76798fb469c8ec9dd2d185572c3)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:48:57.734627 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff5678d7-05a3-489d-b8db-a53714521653 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:48:57.737270 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 105/204] 10.222.0.27 () {56 vars in 1186 bytes} [Fri Aug 5 19:48:57 2022] GET /identity/v3/projects/6f67d76798fb469c8ec9dd2d185572c3 => generated 298 bytes in 329 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:48:57.744281 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5678d7-05a3-489d-b8db-a53714521653 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:48:57.801110 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 100/205] 10.222.0.27 () {58 vars in 1282 bytes} [Fri Aug 5 19:48:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 813 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 05 19:48:57.806653 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408db24a-7626-4f07-80c7-0910cd42453b None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:48:57.896880 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:48:57.940692 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408db24a-7626-4f07-80c7-0910cd42453b None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:48:57.950333 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5678d7-05a3-489d-b8db-a53714521653 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:48:57.988666 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d59ca0d-b7eb-4020-9ba0-471875c4b8dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:48:57.989383 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d59ca0d-b7eb-4020-9ba0-471875c4b8dc None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:48:57.989664 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d59ca0d-b7eb-4020-9ba0-471875c4b8dc None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:48:57.989938 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d59ca0d-b7eb-4020-9ba0-471875c4b8dc None admin] PATH_INFO: `/v3/projects/6440bf75a9134de7ba2c5fb9c23955e2` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:48:57.994330 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d59ca0d-b7eb-4020-9ba0-471875c4b8dc None admin] RBAC: Authorizing `identity:get_project(project_id=6440bf75a9134de7ba2c5fb9c23955e2)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:48:57.996080 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d59ca0d-b7eb-4020-9ba0-471875c4b8dc None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:48:57.999580 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 101/206] 10.222.0.27 () {56 vars in 1186 bytes} [Fri Aug 5 19:48:57 2022] GET /identity/v3/projects/6440bf75a9134de7ba2c5fb9c23955e2 => generated 302 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:48:58.004434 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d59ca0d-b7eb-4020-9ba0-471875c4b8dc None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:48:58.029928 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b35779d-835a-412a-a877-5c418e9996f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:48:58.030564 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b35779d-835a-412a-a877-5c418e9996f0 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:48:58.030841 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b35779d-835a-412a-a877-5c418e9996f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:48:58.031109 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b35779d-835a-412a-a877-5c418e9996f0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:48:58.031946 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b35779d-835a-412a-a877-5c418e9996f0 None admin] RBAC: Adding query filter params (role.id=b885e5d9af42483799785b59b26b6207, scope.project.id=6f67d76798fb469c8ec9dd2d185572c3, user.id=f61db23dcfba4b319e8809ebabda24d7) {{(pid=80364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:48:58.032256 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b35779d-835a-412a-a877-5c418e9996f0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:48:58.035288 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b35779d-835a-412a-a877-5c418e9996f0 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:48:58.049657 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 106/207] 10.222.0.27 () {56 vars in 1374 bytes} [Fri Aug 5 19:48:57 2022] GET /identity/v3/role_assignments?user.id=f61db23dcfba4b319e8809ebabda24d7&scope.project.id=6f67d76798fb469c8ec9dd2d185572c3&role.id=b885e5d9af42483799785b59b26b6207 => generated 608 bytes in 306 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 19:48:58.176680 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d59ca0d-b7eb-4020-9ba0-471875c4b8dc None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:48:58.216355 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:48:58.225991 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d301d35d-b641-4970-af4f-e36aa254e1e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:48:58.227037 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d301d35d-b641-4970-af4f-e36aa254e1e4 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:48:58.227814 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d301d35d-b641-4970-af4f-e36aa254e1e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:48:58.228384 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d301d35d-b641-4970-af4f-e36aa254e1e4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:48:58.229905 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d301d35d-b641-4970-af4f-e36aa254e1e4 None admin] RBAC: Adding query filter params (role.id=dde0aa8028794132913bcba4d91858b8, scope.project.id=6440bf75a9134de7ba2c5fb9c23955e2, user.id=aef034fd1a4f42618eb52ccce58d0014) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:48:58.230217 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d301d35d-b641-4970-af4f-e36aa254e1e4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:48:58.231706 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d301d35d-b641-4970-af4f-e36aa254e1e4 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:48:58.241563 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 102/208] 10.222.0.27 () {56 vars in 1374 bytes} [Fri Aug 5 19:48:58 2022] GET /identity/v3/role_assignments?user.id=aef034fd1a4f42618eb52ccce58d0014&scope.project.id=6440bf75a9134de7ba2c5fb9c23955e2&role.id=dde0aa8028794132913bcba4d91858b8 => generated 259 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 19:48:58.260832 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8499a4e3-8c98-4a40-971e-81cef4a4f9a0 None None] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:48:58.261341 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8499a4e3-8c98-4a40-971e-81cef4a4f9a0 None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:48:58.261511 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2c8603-1c70-481d-bafa-7f02a3945c55 None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:48:58.261959 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2c8603-1c70-481d-bafa-7f02a3945c55 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:48:58.262033 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8499a4e3-8c98-4a40-971e-81cef4a4f9a0 None None] PATH_INFO: `/` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:48:58.262099 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2c8603-1c70-481d-bafa-7f02a3945c55 None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:48:58.263829 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 103/209] 10.222.0.27 () {54 vars in 927 bytes} [Fri Aug 5 19:48:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 05 19:48:58.266870 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 107/210] 10.222.0.27 () {54 vars in 927 bytes} [Fri Aug 5 19:48:58 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:48:58.269500 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16aa8dc-6075-41b5-9932-e4fb367abb15 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:48:58.269795 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16aa8dc-6075-41b5-9932-e4fb367abb15 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:48:58.270062 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16aa8dc-6075-41b5-9932-e4fb367abb15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:48:58.275399 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d47867-0ad4-4f2a-83fd-79cd72961d3f None None] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:48:58.275885 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d47867-0ad4-4f2a-83fd-79cd72961d3f None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:48:58.276341 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d47867-0ad4-4f2a-83fd-79cd72961d3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:48:58.329185 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-c16aa8dc-6075-41b5-9932-e4fb367abb15 None None] MFA Rules not processed for user `03d1834a64cd4a43829827cb99a72747`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:48:58.335556 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-c16aa8dc-6075-41b5-9932-e4fb367abb15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:48:58.341227 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.auth.core [None req-e2d47867-0ad4-4f2a-83fd-79cd72961d3f None None] MFA Rules not processed for user `03d1834a64cd4a43829827cb99a72747`. Rule list: `[]` (Enabled: `True`). {{(pid=80364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:48:58.343322 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-e2d47867-0ad4-4f2a-83fd-79cd72961d3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:48:58.397138 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:48:58.463427 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 104/211] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Aug 5 19:48:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Aug 05 19:48:58.471483 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 108/212] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Aug 5 19:48:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 19:48:58.489415 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16aa8dc-6075-41b5-9932-e4fb367abb15 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:48:58.498740 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d47867-0ad4-4f2a-83fd-79cd72961d3f None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:48:58.612364 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16aa8dc-6075-41b5-9932-e4fb367abb15 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:48:58.637669 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:48:58.643065 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d47867-0ad4-4f2a-83fd-79cd72961d3f None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:48:58.660419 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f2c4f3-6dfb-4945-9acb-06146820aaec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:48:58.661917 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f2c4f3-6dfb-4945-9acb-06146820aaec None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:48:58.662605 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f2c4f3-6dfb-4945-9acb-06146820aaec None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:48:58.663264 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f2c4f3-6dfb-4945-9acb-06146820aaec None admin] PATH_INFO: `/v3/roles/member` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:48:58.679811 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f2c4f3-6dfb-4945-9acb-06146820aaec None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 19:48:58.686076 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-c7f2c4f3-6dfb-4945-9acb-06146820aaec None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 05 19:48:58.686076 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:48:58.686076 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:48:58.686076 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:48:58.686076 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:48:58.686076 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:48:58.686076 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:48:58.686076 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:48:58.686076 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:48:58.686076 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:48:58.686076 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:48:58.686076 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:48:58.686076 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 19:48:58.686076 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 19:48:58.686076 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 19:48:58.686076 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 19:48:58.686986 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 19:48:58.686986 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 19:48:58.686986 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 19:48:58.686986 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 19:48:58.686986 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:48:58.686986 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:48:58.686986 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:48:58.686986 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:48:58.686986 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:48:58.686986 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:48:58.686986 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:48:58.686986 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Aug 05 19:48:58.686986 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:48:58.686986 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:48:58.686986 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:48:58.686986 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:48:58.688326 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:48:58.688326 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:48:58.688326 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:48:58.688326 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:48:58.688326 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 19:48:58.688326 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 19:48:58.688326 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 19:48:58.688326 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 19:48:58.688326 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 19:48:58.688326 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 19:48:58.688326 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 05 19:48:58.688326 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Aug 05 19:48:58.690654 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03be9fee-6afc-4762-ad80-368ed1e5b3be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:48:58.691165 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 105/213] 10.222.0.27 () {56 vars in 1099 bytes} [Fri Aug 5 19:48:58 2022] GET /identity/v3/roles/member => generated 84 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:48:58.696630 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03be9fee-6afc-4762-ad80-368ed1e5b3be None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:48:58.696630 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03be9fee-6afc-4762-ad80-368ed1e5b3be None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:48:58.696630 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03be9fee-6afc-4762-ad80-368ed1e5b3be None admin] PATH_INFO: `/v3/domains/default` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:48:58.697761 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03be9fee-6afc-4762-ad80-368ed1e5b3be None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:48:58.699759 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03be9fee-6afc-4762-ad80-368ed1e5b3be None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:48:58.699940 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f2c4f3-6dfb-4945-9acb-06146820aaec None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:48:58.701950 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 109/214] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Aug 5 19:48:58 2022] GET /identity/v3/domains/default => generated 199 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 19:48:58.708672 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03be9fee-6afc-4762-ad80-368ed1e5b3be None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:48:58.748648 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:48:58.833961 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03be9fee-6afc-4762-ad80-368ed1e5b3be None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:48:58.847440 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f2c4f3-6dfb-4945-9acb-06146820aaec None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:48:58.893046 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c483cc-7e44-4790-a835-4de1e64d160c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:48:58.893553 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd098ba-18cb-4b48-ae69-b7772aa26faa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:48:58.894378 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c483cc-7e44-4790-a835-4de1e64d160c None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:48:58.894748 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd098ba-18cb-4b48-ae69-b7772aa26faa None admin] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:48:58.895031 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c483cc-7e44-4790-a835-4de1e64d160c None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:48:58.895346 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd098ba-18cb-4b48-ae69-b7772aa26faa None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:48:58.895729 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c483cc-7e44-4790-a835-4de1e64d160c None admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:48:58.895856 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd098ba-18cb-4b48-ae69-b7772aa26faa None admin] PATH_INFO: `/v3/users` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:48:58.896822 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c483cc-7e44-4790-a835-4de1e64d160c None admin] RBAC: Adding query filter params (name=member) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:48:58.897133 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd098ba-18cb-4b48-ae69-b7772aa26faa None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:48:58.897493 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c483cc-7e44-4790-a835-4de1e64d160c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:48:58.899090 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c483cc-7e44-4790-a835-4de1e64d160c None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:48:58.900183 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd098ba-18cb-4b48-ae69-b7772aa26faa None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:48:58.911052 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 106/215] 10.222.0.27 () {56 vars in 1101 bytes} [Fri Aug 5 19:48:58 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 19:48:58.920129 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c483cc-7e44-4790-a835-4de1e64d160c None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:48:58.940532 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 110/216] 10.222.0.27 () {60 vars in 1132 bytes} [Fri Aug 5 19:48:58 2022] POST /identity/v3/users => generated 297 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 05 19:48:58.988387 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:48:59.061968 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c483cc-7e44-4790-a835-4de1e64d160c None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:48:59.102018 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f519008f-a455-487c-80f3-d41561956526 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:48:59.102798 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f519008f-a455-487c-80f3-d41561956526 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:48:59.103277 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f519008f-a455-487c-80f3-d41561956526 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:48:59.103740 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f519008f-a455-487c-80f3-d41561956526 None admin] PATH_INFO: `/v3/users/9f8c69917bf44d1aaa26ac61b516a598` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:48:59.122290 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f519008f-a455-487c-80f3-d41561956526 None admin] RBAC: Authorizing `identity:get_user(user_id=9f8c69917bf44d1aaa26ac61b516a598)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:48:59.124247 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f519008f-a455-487c-80f3-d41561956526 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:48:59.126927 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 107/217] 10.222.0.27 () {56 vars in 1177 bytes} [Fri Aug 5 19:48:58 2022] GET /identity/v3/users/9f8c69917bf44d1aaa26ac61b516a598 => generated 297 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 19:48:59.132454 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd098ba-18cb-4b48-ae69-b7772aa26faa None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:48:59.232963 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:48:59.233149 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69994d9d-9e09-4bef-8850-178152fbe512 None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:48:59.233369 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69994d9d-9e09-4bef-8850-178152fbe512 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:48:59.233584 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69994d9d-9e09-4bef-8850-178152fbe512 None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:48:59.235382 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 108/218] 10.222.0.27 () {54 vars in 927 bytes} [Fri Aug 5 19:48:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:48:59.240766 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f3ec47-24f3-48ca-82d7-e4fb0fc02730 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:48:59.241072 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f3ec47-24f3-48ca-82d7-e4fb0fc02730 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:48:59.241343 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f3ec47-24f3-48ca-82d7-e4fb0fc02730 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:48:59.253559 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd098ba-18cb-4b48-ae69-b7772aa26faa None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:48:59.263691 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-34f3ec47-24f3-48ca-82d7-e4fb0fc02730 None None] MFA Rules not processed for user `03d1834a64cd4a43829827cb99a72747`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:48:59.264177 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-34f3ec47-24f3-48ca-82d7-e4fb0fc02730 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:48:59.294971 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3345d260-a279-43ad-8ea1-28ceb75ca48b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:48:59.295613 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3345d260-a279-43ad-8ea1-28ceb75ca48b None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:48:59.295905 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3345d260-a279-43ad-8ea1-28ceb75ca48b None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:48:59.296171 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3345d260-a279-43ad-8ea1-28ceb75ca48b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:48:59.361899 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 109/219] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Aug 5 19:48:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 05 19:48:59.386817 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3345d260-a279-43ad-8ea1-28ceb75ca48b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:48:59.389363 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3345d260-a279-43ad-8ea1-28ceb75ca48b None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:48:59.391413 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f3ec47-24f3-48ca-82d7-e4fb0fc02730 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:48:59.432561 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:48:59.519795 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f3ec47-24f3-48ca-82d7-e4fb0fc02730 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:48:59.529415 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:48:59.559632 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183ccc35-5e2e-480e-b108-57b2d4f9a97a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:48:59.560294 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183ccc35-5e2e-480e-b108-57b2d4f9a97a None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:48:59.560602 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183ccc35-5e2e-480e-b108-57b2d4f9a97a None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:48:59.560881 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183ccc35-5e2e-480e-b108-57b2d4f9a97a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:48:59.567969 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 111/220] 10.222.0.27 () {58 vars in 1282 bytes} [Fri Aug 5 19:48:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 436 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 05 19:48:59.572317 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183ccc35-5e2e-480e-b108-57b2d4f9a97a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 19:48:59.573688 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3345d260-a279-43ad-8ea1-28ceb75ca48b None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:48:59.577467 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-183ccc35-5e2e-480e-b108-57b2d4f9a97a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 19:48:59.577467 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:48:59.577467 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:48:59.577467 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:48:59.577467 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:48:59.577467 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:48:59.577467 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:48:59.577467 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:48:59.577467 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:48:59.577467 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:48:59.577467 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:48:59.577467 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:48:59.577467 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 19:48:59.577467 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 19:48:59.577467 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 19:48:59.577467 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 19:48:59.578356 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 19:48:59.578356 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 19:48:59.578356 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 19:48:59.578356 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 19:48:59.578356 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:48:59.578356 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:48:59.578356 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:48:59.578356 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:48:59.578356 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:48:59.578356 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:48:59.578356 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:48:59.578356 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Aug 05 19:48:59.578356 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:48:59.578356 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:48:59.578356 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:48:59.578356 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:48:59.579745 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:48:59.579745 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:48:59.579745 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:48:59.579745 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:48:59.579745 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 19:48:59.579745 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 19:48:59.579745 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 19:48:59.579745 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 19:48:59.579745 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 19:48:59.579745 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 19:48:59.579745 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 19:48:59.579745 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Aug 05 19:48:59.583519 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 110/221] 10.222.0.27 () {56 vars in 1096 bytes} [Fri Aug 5 19:48:59 2022] GET /identity/v3/roles/admin => generated 83 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 05 19:48:59.585392 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183ccc35-5e2e-480e-b108-57b2d4f9a97a None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:48:59.692634 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:48:59.705328 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183ccc35-5e2e-480e-b108-57b2d4f9a97a None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:48:59.749950 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ecad68c-f420-4849-a940-926e8b7bff66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:48:59.750647 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecad68c-f420-4849-a940-926e8b7bff66 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:48:59.750932 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecad68c-f420-4849-a940-926e8b7bff66 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:48:59.751209 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecad68c-f420-4849-a940-926e8b7bff66 None admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:48:59.751943 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ecad68c-f420-4849-a940-926e8b7bff66 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:48:59.752253 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ecad68c-f420-4849-a940-926e8b7bff66 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:48:59.753687 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ecad68c-f420-4849-a940-926e8b7bff66 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:48:59.768239 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 111/222] 10.222.0.27 () {56 vars in 1099 bytes} [Fri Aug 5 19:48:59 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:48:59.769630 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ecad68c-f420-4849-a940-926e8b7bff66 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:48:59.805787 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3345d260-a279-43ad-8ea1-28ceb75ca48b None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:48:59.868701 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68ec1f2-98cf-4acd-97a3-9ffefd2927ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:48:59.869763 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68ec1f2-98cf-4acd-97a3-9ffefd2927ec None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:48:59.871875 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68ec1f2-98cf-4acd-97a3-9ffefd2927ec None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:48:59.872378 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68ec1f2-98cf-4acd-97a3-9ffefd2927ec None admin] PATH_INFO: `/v3/projects/6440bf75a9134de7ba2c5fb9c23955e2` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:48:59.876457 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f68ec1f2-98cf-4acd-97a3-9ffefd2927ec None admin] RBAC: Authorizing `identity:get_project(project_id=6440bf75a9134de7ba2c5fb9c23955e2)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:48:59.878206 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f68ec1f2-98cf-4acd-97a3-9ffefd2927ec None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:48:59.886721 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 112/223] 10.222.0.27 () {56 vars in 1186 bytes} [Fri Aug 5 19:48:59 2022] GET /identity/v3/projects/6440bf75a9134de7ba2c5fb9c23955e2 => generated 302 bytes in 314 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:48:59.892578 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68ec1f2-98cf-4acd-97a3-9ffefd2927ec None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:48:59.930805 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ecad68c-f420-4849-a940-926e8b7bff66 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:48:59.988580 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:00.027969 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924d99ca-9c82-4302-8ffd-0cbf31017137 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:00.029018 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924d99ca-9c82-4302-8ffd-0cbf31017137 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:00.030759 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924d99ca-9c82-4302-8ffd-0cbf31017137 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:00.031245 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924d99ca-9c82-4302-8ffd-0cbf31017137 None admin] PATH_INFO: `/v3/users/aef034fd1a4f42618eb52ccce58d0014` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:00.034356 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-924d99ca-9c82-4302-8ffd-0cbf31017137 None admin] RBAC: Authorizing `identity:get_user(user_id=aef034fd1a4f42618eb52ccce58d0014)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:00.036321 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-924d99ca-9c82-4302-8ffd-0cbf31017137 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:00.040919 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 112/224] 10.222.0.27 () {56 vars in 1177 bytes} [Fri Aug 5 19:48:59 2022] GET /identity/v3/users/aef034fd1a4f42618eb52ccce58d0014 => generated 283 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:49:00.046743 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924d99ca-9c82-4302-8ffd-0cbf31017137 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:00.072124 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68ec1f2-98cf-4acd-97a3-9ffefd2927ec None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:00.108871 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:00.186921 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924d99ca-9c82-4302-8ffd-0cbf31017137 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:00.201254 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb981e6-4cd6-45b6-9ffd-55272cc2564e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:00.201902 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb981e6-4cd6-45b6-9ffd-55272cc2564e None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:00.202241 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb981e6-4cd6-45b6-9ffd-55272cc2564e None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:00.202448 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb981e6-4cd6-45b6-9ffd-55272cc2564e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:00.203264 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb981e6-4cd6-45b6-9ffd-55272cc2564e None admin] RBAC: Adding query filter params (role.id=1a4f694d53054a24b580138b1ba77f3c, scope.project.id=6440bf75a9134de7ba2c5fb9c23955e2, user.id=9f8c69917bf44d1aaa26ac61b516a598) {{(pid=80364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:49:00.204517 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb981e6-4cd6-45b6-9ffd-55272cc2564e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:00.205704 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb981e6-4cd6-45b6-9ffd-55272cc2564e None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:00.226413 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba4d333-b156-4c68-b7e0-559f7c5d0093 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:00.226786 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 113/225] 10.222.0.27 () {56 vars in 1374 bytes} [Fri Aug 5 19:48:59 2022] GET /identity/v3/role_assignments?user.id=9f8c69917bf44d1aaa26ac61b516a598&scope.project.id=6440bf75a9134de7ba2c5fb9c23955e2&role.id=1a4f694d53054a24b580138b1ba77f3c => generated 259 bytes in 334 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 19:49:00.227614 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba4d333-b156-4c68-b7e0-559f7c5d0093 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:00.228095 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba4d333-b156-4c68-b7e0-559f7c5d0093 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:00.228619 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba4d333-b156-4c68-b7e0-559f7c5d0093 None admin] PATH_INFO: `/v3/projects/6440bf75a9134de7ba2c5fb9c23955e2` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:00.230122 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba4d333-b156-4c68-b7e0-559f7c5d0093 None admin] RBAC: Authorizing `identity:get_project(project_id=6440bf75a9134de7ba2c5fb9c23955e2)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:00.231881 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba4d333-b156-4c68-b7e0-559f7c5d0093 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:00.234453 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 113/226] 10.222.0.27 () {56 vars in 1186 bytes} [Fri Aug 5 19:49:00 2022] GET /identity/v3/projects/6440bf75a9134de7ba2c5fb9c23955e2 => generated 302 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:49:00.245768 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb981e6-4cd6-45b6-9ffd-55272cc2564e None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:00.456863 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb981e6-4cd6-45b6-9ffd-55272cc2564e None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:00.493996 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:00.518011 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf60f66a-cd67-45cf-81fb-bb1181e6a113 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:00.518614 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf60f66a-cd67-45cf-81fb-bb1181e6a113 None admin] REQUEST_METHOD: `PUT` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:00.518896 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf60f66a-cd67-45cf-81fb-bb1181e6a113 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:00.519177 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf60f66a-cd67-45cf-81fb-bb1181e6a113 None admin] PATH_INFO: `/v3/projects/6440bf75a9134de7ba2c5fb9c23955e2/users/aef034fd1a4f42618eb52ccce58d0014/roles/c173688a960b46e5be65f0ccd6fbd734` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:00.521859 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf60f66a-cd67-45cf-81fb-bb1181e6a113 None admin] RBAC: Authorizing `identity:create_grant(project_id=6440bf75a9134de7ba2c5fb9c23955e2, user_id=aef034fd1a4f42618eb52ccce58d0014, role_id=c173688a960b46e5be65f0ccd6fbd734)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:00.523323 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf60f66a-cd67-45cf-81fb-bb1181e6a113 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:00.541816 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 114/227] 10.222.0.27 () {58 vars in 1439 bytes} [Fri Aug 5 19:49:00 2022] PUT /identity/v3/projects/6440bf75a9134de7ba2c5fb9c23955e2/users/aef034fd1a4f42618eb52ccce58d0014/roles/c173688a960b46e5be65f0ccd6fbd734 => generated 0 bytes in 297 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 19:49:00.658801 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68870026-a5ae-445a-9c39-0386d489c5fc None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:00.659770 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68870026-a5ae-445a-9c39-0386d489c5fc None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:00.660253 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68870026-a5ae-445a-9c39-0386d489c5fc None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:00.662236 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 114/228] 10.222.0.27 () {54 vars in 927 bytes} [Fri Aug 5 19:49:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:49:00.668564 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d6cf15-6063-4ef2-8095-0fe3f2da1652 None None] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:00.669172 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d6cf15-6063-4ef2-8095-0fe3f2da1652 None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:00.669649 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d6cf15-6063-4ef2-8095-0fe3f2da1652 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:00.689858 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.auth.core [None req-f4d6cf15-6063-4ef2-8095-0fe3f2da1652 None None] MFA Rules not processed for user `03d1834a64cd4a43829827cb99a72747`. Rule list: `[]` (Enabled: `True`). {{(pid=80364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:49:00.691465 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-f4d6cf15-6063-4ef2-8095-0fe3f2da1652 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:49:00.766315 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 115/229] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Aug 5 19:49:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 19:49:00.790317 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68870026-a5ae-445a-9c39-0386d489c5fc None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:00.905398 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:00.917837 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68870026-a5ae-445a-9c39-0386d489c5fc None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:00.958696 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24da1116-6283-44de-8c45-d6e99e7721d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:00.959719 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24da1116-6283-44de-8c45-d6e99e7721d2 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:00.960136 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24da1116-6283-44de-8c45-d6e99e7721d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:00.960544 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24da1116-6283-44de-8c45-d6e99e7721d2 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:00.973750 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24da1116-6283-44de-8c45-d6e99e7721d2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 19:49:00.979443 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-24da1116-6283-44de-8c45-d6e99e7721d2 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 05 19:49:00.979443 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:49:00.979443 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:49:00.979443 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:49:00.979443 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:49:00.979443 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:49:00.979443 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:49:00.979443 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:49:00.979443 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:49:00.979443 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:49:00.979443 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:49:00.979443 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:49:00.979443 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 19:49:00.979443 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 19:49:00.979443 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 19:49:00.979443 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 19:49:00.980340 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 19:49:00.980340 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 19:49:00.980340 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 19:49:00.980340 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 19:49:00.980340 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:49:00.980340 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:49:00.980340 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:49:00.980340 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:49:00.980340 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:49:00.980340 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:49:00.980340 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:49:00.980340 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Aug 05 19:49:00.980340 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:49:00.980340 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:49:00.980340 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:49:00.980340 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:49:00.981156 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:49:00.981156 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:49:00.981156 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:49:00.981156 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:49:00.981156 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 19:49:00.981156 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 19:49:00.981156 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 19:49:00.981156 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 19:49:00.981156 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 19:49:00.981156 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 19:49:00.981156 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 05 19:49:00.981156 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Aug 05 19:49:00.981156 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 115/230] 10.222.0.27 () {56 vars in 1114 bytes} [Fri Aug 5 19:49:00 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 05 19:49:00.989720 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d6cf15-6063-4ef2-8095-0fe3f2da1652 None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:01.108750 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d6cf15-6063-4ef2-8095-0fe3f2da1652 None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:01.125289 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:01.160748 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435f331b-877d-4a1f-bc85-14229763b9c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:01.161382 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435f331b-877d-4a1f-bc85-14229763b9c3 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:01.161667 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435f331b-877d-4a1f-bc85-14229763b9c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:01.161936 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435f331b-877d-4a1f-bc85-14229763b9c3 None admin] PATH_INFO: `/v3/roles` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:01.162666 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435f331b-877d-4a1f-bc85-14229763b9c3 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:49:01.162974 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435f331b-877d-4a1f-bc85-14229763b9c3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:01.164382 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435f331b-877d-4a1f-bc85-14229763b9c3 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:01.178709 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 116/231] 10.222.0.27 () {56 vars in 1111 bytes} [Fri Aug 5 19:49:00 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 19:49:01.184599 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24da1116-6283-44de-8c45-d6e99e7721d2 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:01.314864 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24da1116-6283-44de-8c45-d6e99e7721d2 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:01.352274 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18911f08-00a0-4c28-b2fc-914cc28fbbd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:01.353326 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18911f08-00a0-4c28-b2fc-914cc28fbbd8 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:01.353831 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18911f08-00a0-4c28-b2fc-914cc28fbbd8 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:01.354338 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18911f08-00a0-4c28-b2fc-914cc28fbbd8 None admin] PATH_INFO: `/v3/users/aef034fd1a4f42618eb52ccce58d0014` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:01.355753 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18911f08-00a0-4c28-b2fc-914cc28fbbd8 None admin] RBAC: Authorizing `identity:get_user(user_id=aef034fd1a4f42618eb52ccce58d0014)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:01.357457 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18911f08-00a0-4c28-b2fc-914cc28fbbd8 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:01.360281 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 116/232] 10.222.0.27 () {56 vars in 1177 bytes} [Fri Aug 5 19:49:01 2022] GET /identity/v3/users/aef034fd1a4f42618eb52ccce58d0014 => generated 283 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:49:01.376297 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435f331b-877d-4a1f-bc85-14229763b9c3 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:01.546576 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435f331b-877d-4a1f-bc85-14229763b9c3 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:01.586332 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d501bc-f089-4e46-ac8d-5eb03b793881 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:01.587372 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d501bc-f089-4e46-ac8d-5eb03b793881 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:01.587882 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d501bc-f089-4e46-ac8d-5eb03b793881 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:01.588424 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d501bc-f089-4e46-ac8d-5eb03b793881 None admin] PATH_INFO: `/v3/projects/6440bf75a9134de7ba2c5fb9c23955e2` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:01.589885 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d501bc-f089-4e46-ac8d-5eb03b793881 None admin] RBAC: Authorizing `identity:get_project(project_id=6440bf75a9134de7ba2c5fb9c23955e2)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:01.591558 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d501bc-f089-4e46-ac8d-5eb03b793881 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:01.593505 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 117/233] 10.222.0.27 () {56 vars in 1186 bytes} [Fri Aug 5 19:49:01 2022] GET /identity/v3/projects/6440bf75a9134de7ba2c5fb9c23955e2 => generated 302 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:49:01.605761 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18911f08-00a0-4c28-b2fc-914cc28fbbd8 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:01.633429 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:01.670168 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcaee83b-f7cf-40d6-9ce8-ddcecd7de2f0 None None] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:01.670930 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcaee83b-f7cf-40d6-9ce8-ddcecd7de2f0 None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:01.671413 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcaee83b-f7cf-40d6-9ce8-ddcecd7de2f0 None None] PATH_INFO: `/` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:01.673114 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 118/234] 10.222.0.27 () {54 vars in 927 bytes} [Fri Aug 5 19:49:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:49:01.686638 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c860961-5616-4b91-98f1-e0f6caa615e2 None None] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:01.686976 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c860961-5616-4b91-98f1-e0f6caa615e2 None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:01.687199 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c860961-5616-4b91-98f1-e0f6caa615e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:01.725684 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.auth.core [None req-2c860961-5616-4b91-98f1-e0f6caa615e2 None None] MFA Rules not processed for user `03d1834a64cd4a43829827cb99a72747`. Rule list: `[]` (Enabled: `True`). {{(pid=80364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:49:01.727015 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-2c860961-5616-4b91-98f1-e0f6caa615e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:49:01.743204 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18911f08-00a0-4c28-b2fc-914cc28fbbd8 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:01.780835 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a51eea-67f9-4324-971a-5981e4ffbead None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:01.782018 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a51eea-67f9-4324-971a-5981e4ffbead None admin] REQUEST_METHOD: `PUT` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:01.782527 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a51eea-67f9-4324-971a-5981e4ffbead None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:01.783015 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a51eea-67f9-4324-971a-5981e4ffbead None admin] PATH_INFO: `/v3/projects/6440bf75a9134de7ba2c5fb9c23955e2/users/aef034fd1a4f42618eb52ccce58d0014/roles/dde0aa8028794132913bcba4d91858b8` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:01.785743 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a51eea-67f9-4324-971a-5981e4ffbead None admin] RBAC: Authorizing `identity:create_grant(project_id=6440bf75a9134de7ba2c5fb9c23955e2, user_id=aef034fd1a4f42618eb52ccce58d0014, role_id=dde0aa8028794132913bcba4d91858b8)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:01.787568 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a51eea-67f9-4324-971a-5981e4ffbead None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:01.794576 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:01.808614 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 117/235] 10.222.0.27 () {58 vars in 1439 bytes} [Fri Aug 5 19:49:01 2022] PUT /identity/v3/projects/6440bf75a9134de7ba2c5fb9c23955e2/users/aef034fd1a4f42618eb52ccce58d0014/roles/dde0aa8028794132913bcba4d91858b8 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 19:49:01.862819 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20e29a4-2124-4880-8c25-221644c95b87 None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:01.863885 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20e29a4-2124-4880-8c25-221644c95b87 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:01.864681 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20e29a4-2124-4880-8c25-221644c95b87 None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:01.868003 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 118/236] 10.222.0.27 () {54 vars in 927 bytes} [Fri Aug 5 19:49:01 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:49:01.877414 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1bd623-e37b-48c3-9bfd-a737e894a70a None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:01.877715 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1bd623-e37b-48c3-9bfd-a737e894a70a None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:01.877966 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1bd623-e37b-48c3-9bfd-a737e894a70a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:01.882432 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 119/237] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Aug 5 19:49:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 05 19:49:01.909204 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-ed1bd623-e37b-48c3-9bfd-a737e894a70a None None] MFA Rules not processed for user `03d1834a64cd4a43829827cb99a72747`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:49:01.910339 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-ed1bd623-e37b-48c3-9bfd-a737e894a70a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:49:01.935076 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c860961-5616-4b91-98f1-e0f6caa615e2 None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:02.068811 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 119/238] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Aug 5 19:49:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 19:49:02.096860 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1bd623-e37b-48c3-9bfd-a737e894a70a None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:02.103169 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c860961-5616-4b91-98f1-e0f6caa615e2 None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:02.110972 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:02.154194 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290b40c2-f70e-4d87-8c83-6d90d14134d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:02.155631 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290b40c2-f70e-4d87-8c83-6d90d14134d7 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:02.156357 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290b40c2-f70e-4d87-8c83-6d90d14134d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:02.157370 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290b40c2-f70e-4d87-8c83-6d90d14134d7 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:02.159524 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-290b40c2-f70e-4d87-8c83-6d90d14134d7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:02.161578 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-290b40c2-f70e-4d87-8c83-6d90d14134d7 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:02.164014 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 120/239] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Aug 5 19:49:01 2022] GET /identity/v3/domains/default => generated 199 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:49:02.170128 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290b40c2-f70e-4d87-8c83-6d90d14134d7 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:02.211573 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:02.289938 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290b40c2-f70e-4d87-8c83-6d90d14134d7 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:02.319592 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1bd623-e37b-48c3-9bfd-a737e894a70a None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:02.341178 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6626e25d-da0f-4156-8091-6412cd64dfed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:02.341839 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6626e25d-da0f-4156-8091-6412cd64dfed None admin] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:02.342365 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6626e25d-da0f-4156-8091-6412cd64dfed None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:02.342645 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6626e25d-da0f-4156-8091-6412cd64dfed None admin] PATH_INFO: `/v3/users` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:02.343356 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6626e25d-da0f-4156-8091-6412cd64dfed None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:02.345544 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6626e25d-da0f-4156-8091-6412cd64dfed None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:02.390215 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7ef723-a2d7-49f5-9b85-f9f769cec1db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:02.390928 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 121/240] 10.222.0.27 () {60 vars in 1132 bytes} [Fri Aug 5 19:49:02 2022] POST /identity/v3/users => generated 293 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 19:49:02.391606 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7ef723-a2d7-49f5-9b85-f9f769cec1db None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:02.391969 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7ef723-a2d7-49f5-9b85-f9f769cec1db None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:02.392258 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7ef723-a2d7-49f5-9b85-f9f769cec1db None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:02.407257 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae7ef723-a2d7-49f5-9b85-f9f769cec1db None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 19:49:02.412918 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-ae7ef723-a2d7-49f5-9b85-f9f769cec1db None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 05 19:49:02.412918 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:49:02.412918 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:49:02.412918 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:49:02.412918 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:49:02.412918 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:49:02.412918 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:49:02.412918 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:49:02.412918 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:49:02.412918 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:49:02.412918 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:49:02.412918 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:49:02.412918 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 19:49:02.412918 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 19:49:02.412918 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 19:49:02.412918 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 19:49:02.413819 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 19:49:02.413819 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 19:49:02.413819 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 19:49:02.413819 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 19:49:02.413819 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:49:02.413819 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:49:02.413819 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:49:02.413819 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:49:02.413819 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:49:02.413819 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:49:02.413819 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:49:02.413819 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Aug 05 19:49:02.413819 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:49:02.413819 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:49:02.413819 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:49:02.413819 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:49:02.414814 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:49:02.414814 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:49:02.414814 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:49:02.414814 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:49:02.414814 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 19:49:02.414814 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 19:49:02.414814 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 19:49:02.414814 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 19:49:02.414814 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 19:49:02.414814 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 19:49:02.414814 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 05 19:49:02.414814 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Aug 05 19:49:02.414814 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 120/241] 10.222.0.27 () {56 vars in 1099 bytes} [Fri Aug 5 19:49:02 2022] GET /identity/v3/roles/reader => generated 84 bytes in 318 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 05 19:49:02.420978 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6626e25d-da0f-4156-8091-6412cd64dfed None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:02.528044 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:02.539193 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6626e25d-da0f-4156-8091-6412cd64dfed None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:02.577406 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77f8b2a-930e-4466-aa4c-b49a12a81a43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:02.578183 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77f8b2a-930e-4466-aa4c-b49a12a81a43 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:02.578903 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77f8b2a-930e-4466-aa4c-b49a12a81a43 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:02.579182 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77f8b2a-930e-4466-aa4c-b49a12a81a43 None admin] PATH_INFO: `/v3/roles` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:02.579924 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c77f8b2a-930e-4466-aa4c-b49a12a81a43 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:49:02.580229 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c77f8b2a-930e-4466-aa4c-b49a12a81a43 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:02.581746 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c77f8b2a-930e-4466-aa4c-b49a12a81a43 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:02.591703 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 122/242] 10.222.0.27 () {56 vars in 1101 bytes} [Fri Aug 5 19:49:02 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:49:02.598624 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7ef723-a2d7-49f5-9b85-f9f769cec1db None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:02.671722 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:02.707627 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7ef723-a2d7-49f5-9b85-f9f769cec1db None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:02.743934 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c201fb67-1ee0-42b5-9bb4-75abd83c27af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:02.744692 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c201fb67-1ee0-42b5-9bb4-75abd83c27af None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:02.744895 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c201fb67-1ee0-42b5-9bb4-75abd83c27af None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:02.745208 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c201fb67-1ee0-42b5-9bb4-75abd83c27af None admin] PATH_INFO: `/v3/users/c5c7582de1584235b7891d319ef5761a` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:02.759651 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c201fb67-1ee0-42b5-9bb4-75abd83c27af None admin] RBAC: Authorizing `identity:get_user(user_id=c5c7582de1584235b7891d319ef5761a)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:02.761359 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c201fb67-1ee0-42b5-9bb4-75abd83c27af None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:02.763369 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 121/243] 10.222.0.27 () {56 vars in 1177 bytes} [Fri Aug 5 19:49:02 2022] GET /identity/v3/users/c5c7582de1584235b7891d319ef5761a => generated 297 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:49:02.769532 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77f8b2a-930e-4466-aa4c-b49a12a81a43 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:02.819012 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882ff4a0-98bf-4d3c-b076-4e321c0d7f58 None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:02.819329 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882ff4a0-98bf-4d3c-b076-4e321c0d7f58 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:02.819555 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882ff4a0-98bf-4d3c-b076-4e321c0d7f58 None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:02.821293 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 122/244] 10.222.0.27 () {54 vars in 927 bytes} [Fri Aug 5 19:49:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 05 19:49:02.826532 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1e0524-d851-4059-87b9-27e301908485 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:02.826808 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1e0524-d851-4059-87b9-27e301908485 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:02.827079 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1e0524-d851-4059-87b9-27e301908485 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:02.850551 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-6f1e0524-d851-4059-87b9-27e301908485 None None] MFA Rules not processed for user `03d1834a64cd4a43829827cb99a72747`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:49:02.851786 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-6f1e0524-d851-4059-87b9-27e301908485 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:49:02.883238 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77f8b2a-930e-4466-aa4c-b49a12a81a43 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:02.930508 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30eb821f-a2f0-4a8d-9fd8-89e771e44001 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:02.931053 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30eb821f-a2f0-4a8d-9fd8-89e771e44001 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:02.931343 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30eb821f-a2f0-4a8d-9fd8-89e771e44001 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:02.931612 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30eb821f-a2f0-4a8d-9fd8-89e771e44001 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:02.939374 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 123/245] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Aug 5 19:49:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 05 19:49:02.962384 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1e0524-d851-4059-87b9-27e301908485 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:02.972901 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:03.015999 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30eb821f-a2f0-4a8d-9fd8-89e771e44001 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:03.017536 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30eb821f-a2f0-4a8d-9fd8-89e771e44001 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:03.027378 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:03.076824 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1e0524-d851-4059-87b9-27e301908485 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:03.118723 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7175d1b6-5e10-4950-9812-2930e9791d69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:03.119690 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7175d1b6-5e10-4950-9812-2930e9791d69 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:03.120163 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7175d1b6-5e10-4950-9812-2930e9791d69 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:03.120667 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7175d1b6-5e10-4950-9812-2930e9791d69 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:03.130823 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 123/246] 10.222.0.27 () {58 vars in 1282 bytes} [Fri Aug 5 19:49:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 05 19:49:03.136621 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30eb821f-a2f0-4a8d-9fd8-89e771e44001 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:03.136940 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7175d1b6-5e10-4950-9812-2930e9791d69 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 19:49:03.142036 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-7175d1b6-5e10-4950-9812-2930e9791d69 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 05 19:49:03.142036 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:49:03.142036 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:49:03.142036 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:49:03.142036 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:49:03.142036 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:49:03.142036 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:49:03.142036 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:49:03.142036 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:49:03.142036 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:49:03.142036 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:49:03.142036 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:49:03.142036 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 19:49:03.142036 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 19:49:03.142036 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 19:49:03.142036 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 19:49:03.144820 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 19:49:03.144820 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 19:49:03.144820 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 19:49:03.144820 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 19:49:03.144820 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:49:03.144820 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:49:03.144820 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:49:03.144820 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:49:03.144820 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:49:03.144820 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:49:03.144820 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:49:03.144820 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Aug 05 19:49:03.144820 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:49:03.144820 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:49:03.144820 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:49:03.144820 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:49:03.146809 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:49:03.146809 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:49:03.146809 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:49:03.146809 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:49:03.146809 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 19:49:03.146809 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 19:49:03.146809 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 19:49:03.146809 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 19:49:03.146809 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 19:49:03.146809 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 19:49:03.146809 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 05 19:49:03.146809 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Aug 05 19:49:03.146809 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 124/247] 10.222.0.27 () {56 vars in 1099 bytes} [Fri Aug 5 19:49:02 2022] GET /identity/v3/roles/member => generated 84 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:49:03.151995 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7175d1b6-5e10-4950-9812-2930e9791d69 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:03.216954 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:03.277600 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30eb821f-a2f0-4a8d-9fd8-89e771e44001 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:03.302954 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7175d1b6-5e10-4950-9812-2930e9791d69 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:03.322020 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6ce3f6-881a-4392-88ac-b4b5fb33cbbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:03.322853 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6ce3f6-881a-4392-88ac-b4b5fb33cbbf None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:03.323137 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6ce3f6-881a-4392-88ac-b4b5fb33cbbf None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:03.323414 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6ce3f6-881a-4392-88ac-b4b5fb33cbbf None admin] PATH_INFO: `/v3/projects/6440bf75a9134de7ba2c5fb9c23955e2` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:03.325063 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b6ce3f6-881a-4392-88ac-b4b5fb33cbbf None admin] RBAC: Authorizing `identity:get_project(project_id=6440bf75a9134de7ba2c5fb9c23955e2)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:03.327202 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b6ce3f6-881a-4392-88ac-b4b5fb33cbbf None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:03.327611 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:03.329933 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 124/248] 10.222.0.27 () {56 vars in 1186 bytes} [Fri Aug 5 19:49:03 2022] GET /identity/v3/projects/6440bf75a9134de7ba2c5fb9c23955e2 => generated 302 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 19:49:03.336059 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6ce3f6-881a-4392-88ac-b4b5fb33cbbf None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:03.370217 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e1f7d8-a654-4351-8e92-784f9f925251 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:03.370217 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e1f7d8-a654-4351-8e92-784f9f925251 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:03.370217 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e1f7d8-a654-4351-8e92-784f9f925251 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:03.370217 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e1f7d8-a654-4351-8e92-784f9f925251 None admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:03.371101 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e1f7d8-a654-4351-8e92-784f9f925251 None admin] RBAC: Adding query filter params (name=member) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:49:03.371101 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e1f7d8-a654-4351-8e92-784f9f925251 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:03.371101 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e1f7d8-a654-4351-8e92-784f9f925251 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:03.404461 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 125/249] 10.222.0.27 () {56 vars in 1101 bytes} [Fri Aug 5 19:49:03 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:49:03.411736 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e1f7d8-a654-4351-8e92-784f9f925251 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:03.557797 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6ce3f6-881a-4392-88ac-b4b5fb33cbbf None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:03.608873 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:03.623221 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e1f7d8-a654-4351-8e92-784f9f925251 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:03.635375 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e695904-d218-4fae-b909-27ec13009d1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:03.636009 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e695904-d218-4fae-b909-27ec13009d1d None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:03.636348 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e695904-d218-4fae-b909-27ec13009d1d None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:03.636714 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e695904-d218-4fae-b909-27ec13009d1d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:03.638098 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e695904-d218-4fae-b909-27ec13009d1d None admin] RBAC: Adding query filter params (role.id=b885e5d9af42483799785b59b26b6207, scope.project.id=6440bf75a9134de7ba2c5fb9c23955e2, user.id=c5c7582de1584235b7891d319ef5761a) {{(pid=80364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:49:03.639075 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e695904-d218-4fae-b909-27ec13009d1d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:03.643779 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e695904-d218-4fae-b909-27ec13009d1d None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:03.654957 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 125/250] 10.222.0.27 () {56 vars in 1374 bytes} [Fri Aug 5 19:49:03 2022] GET /identity/v3/role_assignments?user.id=c5c7582de1584235b7891d319ef5761a&scope.project.id=6440bf75a9134de7ba2c5fb9c23955e2&role.id=b885e5d9af42483799785b59b26b6207 => generated 259 bytes in 318 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 19:49:03.683899 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935159a0-61a4-4584-83b1-8044827550e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:03.684607 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935159a0-61a4-4584-83b1-8044827550e9 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:03.684894 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935159a0-61a4-4584-83b1-8044827550e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:03.685248 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935159a0-61a4-4584-83b1-8044827550e9 None admin] PATH_INFO: `/v3/users/9f8c69917bf44d1aaa26ac61b516a598` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:03.686582 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935159a0-61a4-4584-83b1-8044827550e9 None admin] RBAC: Authorizing `identity:get_user(user_id=9f8c69917bf44d1aaa26ac61b516a598)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:03.688702 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935159a0-61a4-4584-83b1-8044827550e9 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:03.690725 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 126/251] 10.222.0.27 () {56 vars in 1177 bytes} [Fri Aug 5 19:49:03 2022] GET /identity/v3/users/9f8c69917bf44d1aaa26ac61b516a598 => generated 297 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 19:49:03.696371 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e695904-d218-4fae-b909-27ec13009d1d None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:03.742957 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c40fae1-394d-4b9e-894d-942a9f0ffc75 None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:03.743808 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c40fae1-394d-4b9e-894d-942a9f0ffc75 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:03.748517 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c40fae1-394d-4b9e-894d-942a9f0ffc75 None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:03.752339 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 127/252] 10.222.0.27 () {54 vars in 927 bytes} [Fri Aug 5 19:49:03 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 05 19:49:03.761723 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6df0dd0-b2f9-44cf-8ce0-69f52f015cd2 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:03.762342 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6df0dd0-b2f9-44cf-8ce0-69f52f015cd2 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:03.762867 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6df0dd0-b2f9-44cf-8ce0-69f52f015cd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:03.812487 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-b6df0dd0-b2f9-44cf-8ce0-69f52f015cd2 None None] MFA Rules not processed for user `03d1834a64cd4a43829827cb99a72747`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:49:03.814620 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-b6df0dd0-b2f9-44cf-8ce0-69f52f015cd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:49:03.816725 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e695904-d218-4fae-b909-27ec13009d1d None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:03.856124 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9905545d-731b-4094-87b5-29024f90016a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:03.856836 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9905545d-731b-4094-87b5-29024f90016a None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:03.857117 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9905545d-731b-4094-87b5-29024f90016a None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:03.857394 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9905545d-731b-4094-87b5-29024f90016a None admin] PATH_INFO: `/v3/projects/6440bf75a9134de7ba2c5fb9c23955e2` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:03.858752 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9905545d-731b-4094-87b5-29024f90016a None admin] RBAC: Authorizing `identity:get_project(project_id=6440bf75a9134de7ba2c5fb9c23955e2)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:03.860672 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9905545d-731b-4094-87b5-29024f90016a None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:03.862516 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 126/253] 10.222.0.27 () {56 vars in 1186 bytes} [Fri Aug 5 19:49:03 2022] GET /identity/v3/projects/6440bf75a9134de7ba2c5fb9c23955e2 => generated 302 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:49:03.868653 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9905545d-731b-4094-87b5-29024f90016a None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:03.936945 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:03.952554 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:03.988846 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9905545d-731b-4094-87b5-29024f90016a None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:04.002083 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 128/254] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Aug 5 19:49:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 242 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 05 19:49:04.049299 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd66393b-5084-4c30-b099-c60e891760d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:04.050204 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd66393b-5084-4c30-b099-c60e891760d8 None admin] REQUEST_METHOD: `PUT` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:04.050531 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd66393b-5084-4c30-b099-c60e891760d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:04.050782 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd66393b-5084-4c30-b099-c60e891760d8 None admin] PATH_INFO: `/v3/projects/6440bf75a9134de7ba2c5fb9c23955e2/users/9f8c69917bf44d1aaa26ac61b516a598/roles/1a4f694d53054a24b580138b1ba77f3c` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:04.052764 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd66393b-5084-4c30-b099-c60e891760d8 None admin] RBAC: Authorizing `identity:create_grant(project_id=6440bf75a9134de7ba2c5fb9c23955e2, user_id=9f8c69917bf44d1aaa26ac61b516a598, role_id=1a4f694d53054a24b580138b1ba77f3c)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:04.054303 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd66393b-5084-4c30-b099-c60e891760d8 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:04.080665 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 127/255] 10.222.0.27 () {58 vars in 1439 bytes} [Fri Aug 5 19:49:03 2022] PUT /identity/v3/projects/6440bf75a9134de7ba2c5fb9c23955e2/users/9f8c69917bf44d1aaa26ac61b516a598/roles/1a4f694d53054a24b580138b1ba77f3c => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 05 19:49:04.082733 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6df0dd0-b2f9-44cf-8ce0-69f52f015cd2 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:04.250035 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6df0dd0-b2f9-44cf-8ce0-69f52f015cd2 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:04.302027 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569c3d67-523f-46a4-8316-62522e0d6fb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:04.302704 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569c3d67-523f-46a4-8316-62522e0d6fb8 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:04.302982 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569c3d67-523f-46a4-8316-62522e0d6fb8 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:04.303255 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569c3d67-523f-46a4-8316-62522e0d6fb8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:04.319791 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-569c3d67-523f-46a4-8316-62522e0d6fb8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 19:49:04.324911 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-569c3d67-523f-46a4-8316-62522e0d6fb8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 19:49:04.324911 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:49:04.324911 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:49:04.324911 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:49:04.324911 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:49:04.324911 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:49:04.324911 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:49:04.324911 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:49:04.324911 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:49:04.324911 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:49:04.324911 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:49:04.324911 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:49:04.324911 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 19:49:04.324911 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 19:49:04.324911 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 19:49:04.324911 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 19:49:04.325863 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 19:49:04.325863 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 19:49:04.325863 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 19:49:04.325863 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 19:49:04.325863 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:49:04.325863 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:49:04.325863 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:49:04.325863 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:49:04.325863 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:49:04.325863 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:49:04.325863 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:49:04.325863 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Aug 05 19:49:04.325863 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:49:04.325863 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:49:04.325863 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:49:04.325863 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:49:04.327080 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:49:04.327080 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:49:04.327080 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:49:04.327080 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:49:04.327080 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 19:49:04.327080 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 19:49:04.327080 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 19:49:04.327080 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 19:49:04.327080 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 19:49:04.327080 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 19:49:04.327080 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 19:49:04.327080 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Aug 05 19:49:04.327080 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 129/256] 10.222.0.27 () {56 vars in 1096 bytes} [Fri Aug 5 19:49:04 2022] GET /identity/v3/roles/admin => generated 83 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:49:04.334602 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd66393b-5084-4c30-b099-c60e891760d8 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:04.491246 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd66393b-5084-4c30-b099-c60e891760d8 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:04.516110 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:04.540441 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25344b6e-a484-4cda-9ad1-997d5f1696c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:04.541095 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25344b6e-a484-4cda-9ad1-997d5f1696c6 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:04.541375 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25344b6e-a484-4cda-9ad1-997d5f1696c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:04.541650 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25344b6e-a484-4cda-9ad1-997d5f1696c6 None admin] PATH_INFO: `/v3/roles` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:04.542441 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25344b6e-a484-4cda-9ad1-997d5f1696c6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:49:04.542750 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25344b6e-a484-4cda-9ad1-997d5f1696c6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:04.544150 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25344b6e-a484-4cda-9ad1-997d5f1696c6 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:04.555537 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 128/257] 10.222.0.27 () {56 vars in 1099 bytes} [Fri Aug 5 19:49:04 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:49:04.561206 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569c3d67-523f-46a4-8316-62522e0d6fb8 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:04.641702 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:04.742379 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569c3d67-523f-46a4-8316-62522e0d6fb8 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:04.795104 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efe7ca3-75a6-4aeb-89ba-ac8b717a2147 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:04.796476 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efe7ca3-75a6-4aeb-89ba-ac8b717a2147 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:04.796923 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efe7ca3-75a6-4aeb-89ba-ac8b717a2147 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:04.797908 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efe7ca3-75a6-4aeb-89ba-ac8b717a2147 None admin] PATH_INFO: `/v3/users/aef034fd1a4f42618eb52ccce58d0014` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:04.799323 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5efe7ca3-75a6-4aeb-89ba-ac8b717a2147 None admin] RBAC: Authorizing `identity:get_user(user_id=aef034fd1a4f42618eb52ccce58d0014)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:04.801035 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5efe7ca3-75a6-4aeb-89ba-ac8b717a2147 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:04.802881 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 130/258] 10.222.0.27 () {56 vars in 1177 bytes} [Fri Aug 5 19:49:04 2022] GET /identity/v3/users/aef034fd1a4f42618eb52ccce58d0014 => generated 283 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:49:04.808569 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25344b6e-a484-4cda-9ad1-997d5f1696c6 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:04.835086 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5581910d-120d-40ce-83a2-49f674f43106 None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:04.835605 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5581910d-120d-40ce-83a2-49f674f43106 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:04.836028 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5581910d-120d-40ce-83a2-49f674f43106 None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:04.838032 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 131/259] 10.222.0.27 () {54 vars in 927 bytes} [Fri Aug 5 19:49:04 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 05 19:49:04.844292 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb01278e-eb76-4fdb-8e8c-5095d4f048f4 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:04.844780 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb01278e-eb76-4fdb-8e8c-5095d4f048f4 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:04.845456 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb01278e-eb76-4fdb-8e8c-5095d4f048f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:04.874879 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-bb01278e-eb76-4fdb-8e8c-5095d4f048f4 None None] MFA Rules not processed for user `03d1834a64cd4a43829827cb99a72747`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:49:04.876522 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-bb01278e-eb76-4fdb-8e8c-5095d4f048f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:49:05.023647 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25344b6e-a484-4cda-9ad1-997d5f1696c6 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:05.030555 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 132/260] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Aug 5 19:49:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 19:49:05.077444 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb01278e-eb76-4fdb-8e8c-5095d4f048f4 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:05.079039 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6770f0-fe6b-4770-abfc-5f3766088ebf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:05.079683 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6770f0-fe6b-4770-abfc-5f3766088ebf None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:05.079969 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6770f0-fe6b-4770-abfc-5f3766088ebf None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:05.080249 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6770f0-fe6b-4770-abfc-5f3766088ebf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:05.153564 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:05.166226 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:05.199639 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa6770f0-fe6b-4770-abfc-5f3766088ebf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:05.201177 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa6770f0-fe6b-4770-abfc-5f3766088ebf None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:05.282659 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb01278e-eb76-4fdb-8e8c-5095d4f048f4 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:05.325064 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2e8069-6b4e-4048-a3ee-72bde2658b10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:05.325716 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2e8069-6b4e-4048-a3ee-72bde2658b10 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:05.325919 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 129/261] 10.222.0.27 () {58 vars in 1282 bytes} [Fri Aug 5 19:49:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 518 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 05 19:49:05.330479 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2e8069-6b4e-4048-a3ee-72bde2658b10 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:05.330945 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2e8069-6b4e-4048-a3ee-72bde2658b10 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:05.332861 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6770f0-fe6b-4770-abfc-5f3766088ebf None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:05.349956 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af2e8069-6b4e-4048-a3ee-72bde2658b10 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 19:49:05.355677 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-af2e8069-6b4e-4048-a3ee-72bde2658b10 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 05 19:49:05.355677 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:49:05.355677 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:49:05.355677 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:49:05.355677 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:49:05.355677 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:49:05.355677 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:49:05.355677 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:49:05.355677 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:49:05.355677 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:49:05.355677 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:49:05.355677 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:49:05.355677 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 19:49:05.355677 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 19:49:05.355677 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 19:49:05.355677 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 19:49:05.356580 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 19:49:05.356580 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 19:49:05.356580 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 19:49:05.356580 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 19:49:05.356580 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:49:05.356580 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:49:05.356580 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:49:05.356580 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:49:05.356580 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:49:05.356580 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:49:05.356580 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:49:05.356580 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Aug 05 19:49:05.356580 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:49:05.356580 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:49:05.356580 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:49:05.356580 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:49:05.357372 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:49:05.357372 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:49:05.357372 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:49:05.357372 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:49:05.357372 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 19:49:05.357372 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 19:49:05.357372 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 19:49:05.357372 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 19:49:05.357372 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 19:49:05.357372 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 19:49:05.357372 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 05 19:49:05.357372 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Aug 05 19:49:05.362196 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 133/262] 10.222.0.27 () {56 vars in 1114 bytes} [Fri Aug 5 19:49:05 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 285 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 05 19:49:05.369445 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2e8069-6b4e-4048-a3ee-72bde2658b10 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:05.485883 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:05.534558 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6770f0-fe6b-4770-abfc-5f3766088ebf None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:05.537400 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:05.556377 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2e8069-6b4e-4048-a3ee-72bde2658b10 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:05.625267 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173f9c6d-2d67-4bb0-855e-4a45d2924150 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:05.625921 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173f9c6d-2d67-4bb0-855e-4a45d2924150 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:05.626207 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173f9c6d-2d67-4bb0-855e-4a45d2924150 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:05.626487 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173f9c6d-2d67-4bb0-855e-4a45d2924150 None admin] PATH_INFO: `/v3/projects/6440bf75a9134de7ba2c5fb9c23955e2` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:05.626974 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb94a864-8ba2-430c-833b-f272bdaef398 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:05.627682 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb94a864-8ba2-430c-833b-f272bdaef398 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:05.627968 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb94a864-8ba2-430c-833b-f272bdaef398 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:05.628239 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb94a864-8ba2-430c-833b-f272bdaef398 None admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:05.629144 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb94a864-8ba2-430c-833b-f272bdaef398 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:49:05.629456 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb94a864-8ba2-430c-833b-f272bdaef398 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:05.630837 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb94a864-8ba2-430c-833b-f272bdaef398 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:05.633322 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-173f9c6d-2d67-4bb0-855e-4a45d2924150 None admin] RBAC: Authorizing `identity:get_project(project_id=6440bf75a9134de7ba2c5fb9c23955e2)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:05.634811 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-173f9c6d-2d67-4bb0-855e-4a45d2924150 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:05.636939 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 130/263] 10.222.0.27 () {56 vars in 1186 bytes} [Fri Aug 5 19:49:05 2022] GET /identity/v3/projects/6440bf75a9134de7ba2c5fb9c23955e2 => generated 302 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:49:05.643442 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173f9c6d-2d67-4bb0-855e-4a45d2924150 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:05.653408 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 134/264] 10.222.0.27 () {56 vars in 1111 bytes} [Fri Aug 5 19:49:05 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:49:05.658917 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb94a864-8ba2-430c-833b-f272bdaef398 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:05.847331 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb94a864-8ba2-430c-833b-f272bdaef398 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:05.890283 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173f9c6d-2d67-4bb0-855e-4a45d2924150 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:05.925247 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6adc6093-e59d-4143-9209-dd9465b56068 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:05.928432 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adc6093-e59d-4143-9209-dd9465b56068 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:05.928871 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adc6093-e59d-4143-9209-dd9465b56068 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:05.929150 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adc6093-e59d-4143-9209-dd9465b56068 None admin] PATH_INFO: `/v3/users/aef034fd1a4f42618eb52ccce58d0014` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:05.931341 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6adc6093-e59d-4143-9209-dd9465b56068 None admin] RBAC: Authorizing `identity:get_user(user_id=aef034fd1a4f42618eb52ccce58d0014)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:05.932827 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6adc6093-e59d-4143-9209-dd9465b56068 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:05.934774 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 135/265] 10.222.0.27 () {56 vars in 1177 bytes} [Fri Aug 5 19:49:05 2022] GET /identity/v3/users/aef034fd1a4f42618eb52ccce58d0014 => generated 283 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:49:05.936663 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bfde56-6218-41c0-aab3-591e29e7b071 None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:05.937126 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bfde56-6218-41c0-aab3-591e29e7b071 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:05.938098 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bfde56-6218-41c0-aab3-591e29e7b071 None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:05.939535 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 136/266] 10.222.0.27 () {54 vars in 927 bytes} [Fri Aug 5 19:49:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:49:05.945913 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23afdd4-cc4c-4280-b16d-a5a02cb36f11 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:05.946409 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23afdd4-cc4c-4280-b16d-a5a02cb36f11 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:05.946837 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23afdd4-cc4c-4280-b16d-a5a02cb36f11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:05.973101 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e922df22-0795-45bd-b5e2-8ed3f7d82d0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:05.974299 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e922df22-0795-45bd-b5e2-8ed3f7d82d0e None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:05.974783 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e922df22-0795-45bd-b5e2-8ed3f7d82d0e None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:05.975256 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e922df22-0795-45bd-b5e2-8ed3f7d82d0e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:05.976311 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e922df22-0795-45bd-b5e2-8ed3f7d82d0e None admin] RBAC: Adding query filter params (role.id=c173688a960b46e5be65f0ccd6fbd734, scope.project.id=6440bf75a9134de7ba2c5fb9c23955e2, user.id=aef034fd1a4f42618eb52ccce58d0014) {{(pid=80364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:49:05.977032 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e922df22-0795-45bd-b5e2-8ed3f7d82d0e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:05.978726 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e922df22-0795-45bd-b5e2-8ed3f7d82d0e None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:05.987084 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 131/267] 10.222.0.27 () {56 vars in 1374 bytes} [Fri Aug 5 19:49:05 2022] GET /identity/v3/role_assignments?user.id=aef034fd1a4f42618eb52ccce58d0014&scope.project.id=6440bf75a9134de7ba2c5fb9c23955e2&role.id=c173688a960b46e5be65f0ccd6fbd734 => generated 608 bytes in 344 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:49:05.988530 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-a23afdd4-cc4c-4280-b16d-a5a02cb36f11 None None] MFA Rules not processed for user `03d1834a64cd4a43829827cb99a72747`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:49:05.988822 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e922df22-0795-45bd-b5e2-8ed3f7d82d0e None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:05.990382 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-a23afdd4-cc4c-4280-b16d-a5a02cb36f11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:49:06.003131 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:06.025544 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:06.081022 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 137/268] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Aug 5 19:49:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 19:49:06.104823 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23afdd4-cc4c-4280-b16d-a5a02cb36f11 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:06.133742 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e922df22-0795-45bd-b5e2-8ed3f7d82d0e None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:06.174499 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4c81e0-6bee-443f-91ae-d10e0f13122a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:06.175499 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4c81e0-6bee-443f-91ae-d10e0f13122a None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:06.175982 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4c81e0-6bee-443f-91ae-d10e0f13122a None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:06.176454 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4c81e0-6bee-443f-91ae-d10e0f13122a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:06.257319 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf4c81e0-6bee-443f-91ae-d10e0f13122a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:06.259366 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf4c81e0-6bee-443f-91ae-d10e0f13122a None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:06.271101 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:06.295597 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23afdd4-cc4c-4280-b16d-a5a02cb36f11 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:06.375879 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 132/269] 10.222.0.27 () {58 vars in 1282 bytes} [Fri Aug 5 19:49:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 388 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 05 19:49:06.384617 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4c81e0-6bee-443f-91ae-d10e0f13122a None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:06.437742 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e1fa16-898d-4a18-907e-92f9a7a248a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:06.439249 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e1fa16-898d-4a18-907e-92f9a7a248a2 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:06.439983 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e1fa16-898d-4a18-907e-92f9a7a248a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:06.440747 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e1fa16-898d-4a18-907e-92f9a7a248a2 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:06.486418 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e1fa16-898d-4a18-907e-92f9a7a248a2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 19:49:06.503372 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-51e1fa16-898d-4a18-907e-92f9a7a248a2 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 05 19:49:06.503372 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:49:06.503372 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:49:06.503372 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:49:06.503372 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:49:06.503372 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:49:06.503372 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:49:06.503372 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:49:06.503372 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:49:06.503372 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:49:06.503372 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:49:06.503372 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:49:06.503372 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 19:49:06.503372 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 19:49:06.503372 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 19:49:06.503372 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 19:49:06.505039 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 19:49:06.505039 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 19:49:06.505039 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 19:49:06.505039 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 19:49:06.505039 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:49:06.505039 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:49:06.505039 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:49:06.505039 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:49:06.505039 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:49:06.505039 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:49:06.505039 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:49:06.505039 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Aug 05 19:49:06.505039 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:49:06.505039 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:49:06.505039 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:49:06.505039 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:49:06.506517 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:49:06.506517 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:49:06.506517 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:49:06.506517 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:49:06.506517 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 19:49:06.506517 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 19:49:06.506517 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 19:49:06.506517 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 19:49:06.506517 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 19:49:06.506517 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 19:49:06.506517 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 05 19:49:06.506517 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Aug 05 19:49:06.510309 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 138/270] 10.222.0.27 () {56 vars in 1099 bytes} [Fri Aug 5 19:49:06 2022] GET /identity/v3/roles/member => generated 84 bytes in 406 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 05 19:49:06.515876 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e1fa16-898d-4a18-907e-92f9a7a248a2 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:06.525246 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4c81e0-6bee-443f-91ae-d10e0f13122a None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:06.536425 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:06.566061 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3438c52-2843-4032-af6a-6d699c15b89d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:06.566274 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:06.567445 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3438c52-2843-4032-af6a-6d699c15b89d None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:06.567981 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3438c52-2843-4032-af6a-6d699c15b89d None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:06.568481 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3438c52-2843-4032-af6a-6d699c15b89d None admin] PATH_INFO: `/v3/projects/6440bf75a9134de7ba2c5fb9c23955e2` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:06.570250 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3438c52-2843-4032-af6a-6d699c15b89d None admin] RBAC: Authorizing `identity:get_project(project_id=6440bf75a9134de7ba2c5fb9c23955e2)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:06.571872 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3438c52-2843-4032-af6a-6d699c15b89d None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:06.573782 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 133/271] 10.222.0.27 () {56 vars in 1186 bytes} [Fri Aug 5 19:49:06 2022] GET /identity/v3/projects/6440bf75a9134de7ba2c5fb9c23955e2 => generated 302 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:49:06.583678 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3438c52-2843-4032-af6a-6d699c15b89d None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:06.728573 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e1fa16-898d-4a18-907e-92f9a7a248a2 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:06.744456 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3438c52-2843-4032-af6a-6d699c15b89d None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:06.800195 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7d05fe-ec3a-4119-80f0-5ae77003c569 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:06.801133 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7d05fe-ec3a-4119-80f0-5ae77003c569 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:06.801535 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7d05fe-ec3a-4119-80f0-5ae77003c569 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:06.801950 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7d05fe-ec3a-4119-80f0-5ae77003c569 None admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:06.802740 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc7d05fe-ec3a-4119-80f0-5ae77003c569 None admin] RBAC: Adding query filter params (name=member) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:49:06.803150 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc7d05fe-ec3a-4119-80f0-5ae77003c569 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:06.804651 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f457e6ad-4e6a-474b-aec8-99cb4054b72b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:06.804877 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc7d05fe-ec3a-4119-80f0-5ae77003c569 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:06.805944 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f457e6ad-4e6a-474b-aec8-99cb4054b72b None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:06.806616 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f457e6ad-4e6a-474b-aec8-99cb4054b72b None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:06.807036 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f457e6ad-4e6a-474b-aec8-99cb4054b72b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:06.808359 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f457e6ad-4e6a-474b-aec8-99cb4054b72b None admin] RBAC: Adding query filter params (role.id=dde0aa8028794132913bcba4d91858b8, scope.project.id=6440bf75a9134de7ba2c5fb9c23955e2, user.id=aef034fd1a4f42618eb52ccce58d0014) {{(pid=80364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:49:06.809983 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f457e6ad-4e6a-474b-aec8-99cb4054b72b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:06.812444 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f457e6ad-4e6a-474b-aec8-99cb4054b72b None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:06.816641 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 139/272] 10.222.0.27 () {56 vars in 1101 bytes} [Fri Aug 5 19:49:06 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 301 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 19:49:06.818844 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1289f43-694a-4921-ad85-34a247f7928a None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:06.819417 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1289f43-694a-4921-ad85-34a247f7928a None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:06.820165 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1289f43-694a-4921-ad85-34a247f7928a None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:06.822329 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 140/273] 10.222.0.27 () {54 vars in 927 bytes} [Fri Aug 5 19:49:06 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:49:06.823488 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1289f43-694a-4921-ad85-34a247f7928a None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:06.823642 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 134/274] 10.222.0.27 () {56 vars in 1374 bytes} [Fri Aug 5 19:49:06 2022] GET /identity/v3/role_assignments?user.id=aef034fd1a4f42618eb52ccce58d0014&scope.project.id=6440bf75a9134de7ba2c5fb9c23955e2&role.id=dde0aa8028794132913bcba4d91858b8 => generated 608 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 19:49:06.832834 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840683e6-5f04-408d-8789-e8c47824a3bf None None] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:06.833377 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840683e6-5f04-408d-8789-e8c47824a3bf None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:06.833882 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840683e6-5f04-408d-8789-e8c47824a3bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:06.862436 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.auth.core [None req-840683e6-5f04-408d-8789-e8c47824a3bf None None] MFA Rules not processed for user `03d1834a64cd4a43829827cb99a72747`. Rule list: `[]` (Enabled: `True`). {{(pid=80364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:49:06.864372 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-840683e6-5f04-408d-8789-e8c47824a3bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:49:06.963426 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:06.997170 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:07.027273 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 135/275] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Aug 5 19:49:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 05 19:49:07.029584 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14747a9-c712-47ae-9125-09dd1198cd29 None None] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:07.029885 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14747a9-c712-47ae-9125-09dd1198cd29 None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:07.030163 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14747a9-c712-47ae-9125-09dd1198cd29 None None] PATH_INFO: `/` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:07.031695 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 136/276] 10.222.0.27 () {54 vars in 927 bytes} [Fri Aug 5 19:49:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:49:07.037201 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eae5d2b-03e1-4508-bedc-ace6a74db471 None None] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:07.037480 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eae5d2b-03e1-4508-bedc-ace6a74db471 None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:07.037764 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eae5d2b-03e1-4508-bedc-ace6a74db471 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:07.044777 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1289f43-694a-4921-ad85-34a247f7928a None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:07.062316 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.auth.core [None req-0eae5d2b-03e1-4508-bedc-ace6a74db471 None None] MFA Rules not processed for user `03d1834a64cd4a43829827cb99a72747`. Rule list: `[]` (Enabled: `True`). {{(pid=80364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:49:07.065055 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-0eae5d2b-03e1-4508-bedc-ace6a74db471 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:49:07.091929 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ccd2da-4303-4425-b62a-b251eb52b957 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:07.092985 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ccd2da-4303-4425-b62a-b251eb52b957 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:07.093464 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ccd2da-4303-4425-b62a-b251eb52b957 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:07.093976 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ccd2da-4303-4425-b62a-b251eb52b957 None admin] PATH_INFO: `/v3/users/d43d81977c134189a3fbfc92c8e6b52e` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:07.109923 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ccd2da-4303-4425-b62a-b251eb52b957 None admin] RBAC: Authorizing `identity:get_user(user_id=d43d81977c134189a3fbfc92c8e6b52e)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:07.111934 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ccd2da-4303-4425-b62a-b251eb52b957 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:07.113777 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 141/277] 10.222.0.27 () {56 vars in 1177 bytes} [Fri Aug 5 19:49:06 2022] GET /identity/v3/users/d43d81977c134189a3fbfc92c8e6b52e => generated 293 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:49:07.114572 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ccd2da-4303-4425-b62a-b251eb52b957 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:07.169225 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 137/278] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Aug 5 19:49:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 05 19:49:07.170195 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eae5d2b-03e1-4508-bedc-ace6a74db471 None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:07.270032 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ccd2da-4303-4425-b62a-b251eb52b957 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:07.315415 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7f6498-d833-4fa9-8e17-a612b5a69923 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:07.316060 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7f6498-d833-4fa9-8e17-a612b5a69923 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:07.316334 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7f6498-d833-4fa9-8e17-a612b5a69923 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:07.316642 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7f6498-d833-4fa9-8e17-a612b5a69923 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:07.317839 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc7f6498-d833-4fa9-8e17-a612b5a69923 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:07.319295 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc7f6498-d833-4fa9-8e17-a612b5a69923 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:07.320963 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 142/279] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Aug 5 19:49:07 2022] GET /identity/v3/domains/default => generated 199 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:49:07.321569 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:07.321729 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7f6498-d833-4fa9-8e17-a612b5a69923 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:07.325109 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:07.328995 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eae5d2b-03e1-4508-bedc-ace6a74db471 None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:07.373851 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52fd3ac6-6a57-4f33-8c4e-c79c378ca1b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:07.374500 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fd3ac6-6a57-4f33-8c4e-c79c378ca1b0 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:07.374779 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fd3ac6-6a57-4f33-8c4e-c79c378ca1b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:07.375673 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fd3ac6-6a57-4f33-8c4e-c79c378ca1b0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:07.376479 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52fd3ac6-6a57-4f33-8c4e-c79c378ca1b0 None admin] RBAC: Adding query filter params (role.id=1a4f694d53054a24b580138b1ba77f3c, user.id=d43d81977c134189a3fbfc92c8e6b52e, scope.system=all) {{(pid=80364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:49:07.376848 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52fd3ac6-6a57-4f33-8c4e-c79c378ca1b0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:07.378323 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52fd3ac6-6a57-4f33-8c4e-c79c378ca1b0 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:07.383610 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 138/280] 10.222.0.27 () {56 vars in 1308 bytes} [Fri Aug 5 19:49:07 2022] GET /identity/v3/role_assignments?user.id=d43d81977c134189a3fbfc92c8e6b52e&scope.system=all&role.id=1a4f694d53054a24b580138b1ba77f3c => generated 226 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:49:07.384864 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454f0c09-e5b6-43c4-9b4d-121354ef5f5d None None] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:07.385166 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454f0c09-e5b6-43c4-9b4d-121354ef5f5d None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:07.385460 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454f0c09-e5b6-43c4-9b4d-121354ef5f5d None None] PATH_INFO: `/` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:07.386733 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 139/281] 10.222.0.27 () {54 vars in 927 bytes} [Fri Aug 5 19:49:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:49:07.391260 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454f0c09-e5b6-43c4-9b4d-121354ef5f5d None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:07.447012 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7f6498-d833-4fa9-8e17-a612b5a69923 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:07.490831 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3415150d-65e2-4bb1-a96f-1215b9422315 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:07.491497 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3415150d-65e2-4bb1-a96f-1215b9422315 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:07.491778 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3415150d-65e2-4bb1-a96f-1215b9422315 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:07.492046 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3415150d-65e2-4bb1-a96f-1215b9422315 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:07.507420 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3415150d-65e2-4bb1-a96f-1215b9422315 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 19:49:07.512997 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-3415150d-65e2-4bb1-a96f-1215b9422315 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 05 19:49:07.512997 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:49:07.512997 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:49:07.512997 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:49:07.512997 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:49:07.512997 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:49:07.512997 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:49:07.512997 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:49:07.512997 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:49:07.512997 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:49:07.512997 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:49:07.512997 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:49:07.512997 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 19:49:07.512997 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 19:49:07.512997 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 19:49:07.512997 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 19:49:07.513961 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 19:49:07.513961 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 19:49:07.513961 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 19:49:07.513961 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 19:49:07.513961 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:49:07.513961 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:49:07.513961 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:49:07.513961 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:49:07.513961 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:49:07.513961 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:49:07.513961 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:49:07.513961 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Aug 05 19:49:07.513961 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:49:07.513961 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:49:07.513961 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:49:07.513961 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:49:07.514735 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:49:07.514735 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:49:07.514735 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:49:07.514735 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:49:07.514735 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 19:49:07.514735 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 19:49:07.514735 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 19:49:07.514735 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 19:49:07.514735 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 19:49:07.514735 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 19:49:07.514735 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 05 19:49:07.514735 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Aug 05 19:49:07.514735 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 143/282] 10.222.0.27 () {56 vars in 1099 bytes} [Fri Aug 5 19:49:07 2022] GET /identity/v3/roles/reader => generated 84 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:49:07.516191 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804bc711-d95a-4a47-a94c-3a0fc5a50600 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:07.516191 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804bc711-d95a-4a47-a94c-3a0fc5a50600 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:07.516191 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804bc711-d95a-4a47-a94c-3a0fc5a50600 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:07.524807 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454f0c09-e5b6-43c4-9b4d-121354ef5f5d None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:07.538953 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-804bc711-d95a-4a47-a94c-3a0fc5a50600 None None] MFA Rules not processed for user `03d1834a64cd4a43829827cb99a72747`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:49:07.540205 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-804bc711-d95a-4a47-a94c-3a0fc5a50600 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:49:07.568646 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85cd1373-2261-45d0-ab29-e45523480dfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:07.569268 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85cd1373-2261-45d0-ab29-e45523480dfd None admin] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:07.569550 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85cd1373-2261-45d0-ab29-e45523480dfd None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:07.569823 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85cd1373-2261-45d0-ab29-e45523480dfd None admin] PATH_INFO: `/v3/users` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:07.570511 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85cd1373-2261-45d0-ab29-e45523480dfd None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:07.571990 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85cd1373-2261-45d0-ab29-e45523480dfd None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:07.597207 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 140/283] 10.222.0.27 () {60 vars in 1132 bytes} [Fri Aug 5 19:49:07 2022] POST /identity/v3/users => generated 293 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 19:49:07.608804 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85cd1373-2261-45d0-ab29-e45523480dfd None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:07.625587 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:07.628671 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:07.649040 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 144/284] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Aug 5 19:49:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 19:49:07.673691 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804bc711-d95a-4a47-a94c-3a0fc5a50600 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:07.747261 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85cd1373-2261-45d0-ab29-e45523480dfd None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:07.798216 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bded75b4-43bb-4c78-bdd3-f93e981cf92d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:07.798749 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bded75b4-43bb-4c78-bdd3-f93e981cf92d None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:07.799061 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bded75b4-43bb-4c78-bdd3-f93e981cf92d None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:07.799837 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bded75b4-43bb-4c78-bdd3-f93e981cf92d None admin] PATH_INFO: `/v3/roles` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:07.800600 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bded75b4-43bb-4c78-bdd3-f93e981cf92d None admin] RBAC: Adding query filter params (name=reader) {{(pid=80364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:49:07.800912 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bded75b4-43bb-4c78-bdd3-f93e981cf92d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:07.802687 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bded75b4-43bb-4c78-bdd3-f93e981cf92d None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:07.813936 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 141/285] 10.222.0.27 () {56 vars in 1101 bytes} [Fri Aug 5 19:49:07 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:49:07.819564 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bded75b4-43bb-4c78-bdd3-f93e981cf92d None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:07.833547 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804bc711-d95a-4a47-a94c-3a0fc5a50600 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:07.891160 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64e9817-c209-4f57-a8a1-0f7bb072a179 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:07.891837 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64e9817-c209-4f57-a8a1-0f7bb072a179 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:07.892147 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64e9817-c209-4f57-a8a1-0f7bb072a179 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:07.892740 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64e9817-c209-4f57-a8a1-0f7bb072a179 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:07.908653 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64e9817-c209-4f57-a8a1-0f7bb072a179 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 19:49:07.915288 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-d64e9817-c209-4f57-a8a1-0f7bb072a179 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 05 19:49:07.915288 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:49:07.915288 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:49:07.915288 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:49:07.915288 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:49:07.915288 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:49:07.915288 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:49:07.915288 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:49:07.915288 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:49:07.915288 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:49:07.915288 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:49:07.915288 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:49:07.915288 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 19:49:07.915288 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 19:49:07.915288 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 19:49:07.915288 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 19:49:07.916863 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 19:49:07.916863 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 19:49:07.916863 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 19:49:07.916863 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 19:49:07.916863 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:49:07.916863 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:49:07.916863 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:49:07.916863 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:49:07.916863 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:49:07.916863 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:49:07.916863 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:49:07.916863 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Aug 05 19:49:07.916863 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:49:07.916863 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:49:07.916863 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:49:07.916863 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:49:07.918219 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:49:07.918219 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:49:07.918219 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:49:07.918219 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:49:07.918219 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 19:49:07.918219 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 19:49:07.918219 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 19:49:07.918219 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 19:49:07.918219 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 19:49:07.918219 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 19:49:07.918219 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 05 19:49:07.918219 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Aug 05 19:49:07.920609 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 145/286] 10.222.0.27 () {56 vars in 1099 bytes} [Fri Aug 5 19:49:07 2022] GET /identity/v3/roles/member => generated 84 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:49:07.923209 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64e9817-c209-4f57-a8a1-0f7bb072a179 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:07.959718 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:07.984966 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bded75b4-43bb-4c78-bdd3-f93e981cf92d None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:07.997431 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:08.030766 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981027a0-e770-4d4d-9f08-225fd94adac4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:08.031929 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981027a0-e770-4d4d-9f08-225fd94adac4 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:08.032258 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981027a0-e770-4d4d-9f08-225fd94adac4 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:08.033952 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981027a0-e770-4d4d-9f08-225fd94adac4 None admin] PATH_INFO: `/v3/users/c5c7582de1584235b7891d319ef5761a` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:08.036667 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981027a0-e770-4d4d-9f08-225fd94adac4 None admin] RBAC: Authorizing `identity:get_user(user_id=c5c7582de1584235b7891d319ef5761a)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:08.036667 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981027a0-e770-4d4d-9f08-225fd94adac4 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:08.039933 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 142/287] 10.222.0.27 () {56 vars in 1177 bytes} [Fri Aug 5 19:49:07 2022] GET /identity/v3/users/c5c7582de1584235b7891d319ef5761a => generated 297 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:49:08.044633 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981027a0-e770-4d4d-9f08-225fd94adac4 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:08.067584 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64e9817-c209-4f57-a8a1-0f7bb072a179 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:08.114049 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aecac701-c101-4474-911f-915d0e5a6880 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:08.115729 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecac701-c101-4474-911f-915d0e5a6880 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:08.116355 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecac701-c101-4474-911f-915d0e5a6880 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:08.116943 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecac701-c101-4474-911f-915d0e5a6880 None admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:08.118165 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aecac701-c101-4474-911f-915d0e5a6880 None admin] RBAC: Adding query filter params (name=member) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:49:08.119080 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aecac701-c101-4474-911f-915d0e5a6880 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:08.121589 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aecac701-c101-4474-911f-915d0e5a6880 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:08.135218 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 146/288] 10.222.0.27 () {56 vars in 1101 bytes} [Fri Aug 5 19:49:07 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:49:08.145641 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aecac701-c101-4474-911f-915d0e5a6880 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:08.176597 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981027a0-e770-4d4d-9f08-225fd94adac4 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:08.205799 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5236edc5-93e9-4dda-bcbb-ed09be29e609 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:08.206539 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5236edc5-93e9-4dda-bcbb-ed09be29e609 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:08.206885 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5236edc5-93e9-4dda-bcbb-ed09be29e609 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:08.207226 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5236edc5-93e9-4dda-bcbb-ed09be29e609 None admin] PATH_INFO: `/v3/projects/6440bf75a9134de7ba2c5fb9c23955e2` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:08.208688 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5236edc5-93e9-4dda-bcbb-ed09be29e609 None admin] RBAC: Authorizing `identity:get_project(project_id=6440bf75a9134de7ba2c5fb9c23955e2)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:08.209999 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5236edc5-93e9-4dda-bcbb-ed09be29e609 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:08.211826 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 143/289] 10.222.0.27 () {56 vars in 1186 bytes} [Fri Aug 5 19:49:08 2022] GET /identity/v3/projects/6440bf75a9134de7ba2c5fb9c23955e2 => generated 302 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 19:49:08.217613 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5236edc5-93e9-4dda-bcbb-ed09be29e609 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:08.238459 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:08.270749 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aecac701-c101-4474-911f-915d0e5a6880 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:08.286805 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:08.309978 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a54cfb-dc0a-4f31-b87d-b73c16104c97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:08.310920 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a54cfb-dc0a-4f31-b87d-b73c16104c97 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:08.311408 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a54cfb-dc0a-4f31-b87d-b73c16104c97 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:08.311861 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a54cfb-dc0a-4f31-b87d-b73c16104c97 None admin] PATH_INFO: `/v3/users/9f8c69917bf44d1aaa26ac61b516a598` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:08.313853 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a54cfb-dc0a-4f31-b87d-b73c16104c97 None admin] RBAC: Authorizing `identity:get_user(user_id=9f8c69917bf44d1aaa26ac61b516a598)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:08.316723 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a54cfb-dc0a-4f31-b87d-b73c16104c97 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:08.320598 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 147/290] 10.222.0.27 () {56 vars in 1177 bytes} [Fri Aug 5 19:49:08 2022] GET /identity/v3/users/9f8c69917bf44d1aaa26ac61b516a598 => generated 297 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:49:08.328540 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5236edc5-93e9-4dda-bcbb-ed09be29e609 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:08.333653 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a54cfb-dc0a-4f31-b87d-b73c16104c97 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:08.367169 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211a8f02-a01d-49ad-b747-11cf5b090213 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:08.368234 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211a8f02-a01d-49ad-b747-11cf5b090213 None admin] REQUEST_METHOD: `PUT` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:08.368863 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211a8f02-a01d-49ad-b747-11cf5b090213 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:08.369357 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211a8f02-a01d-49ad-b747-11cf5b090213 None admin] PATH_INFO: `/v3/projects/6440bf75a9134de7ba2c5fb9c23955e2/users/c5c7582de1584235b7891d319ef5761a/roles/b885e5d9af42483799785b59b26b6207` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:08.371760 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-211a8f02-a01d-49ad-b747-11cf5b090213 None admin] RBAC: Authorizing `identity:create_grant(project_id=6440bf75a9134de7ba2c5fb9c23955e2, user_id=c5c7582de1584235b7891d319ef5761a, role_id=b885e5d9af42483799785b59b26b6207)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:08.374162 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-211a8f02-a01d-49ad-b747-11cf5b090213 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:08.391907 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 144/291] 10.222.0.27 () {58 vars in 1439 bytes} [Fri Aug 5 19:49:08 2022] PUT /identity/v3/projects/6440bf75a9134de7ba2c5fb9c23955e2/users/c5c7582de1584235b7891d319ef5761a/roles/b885e5d9af42483799785b59b26b6207 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 19:49:08.539549 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a54cfb-dc0a-4f31-b87d-b73c16104c97 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:08.597660 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea1da8a-39a0-4ff0-8603-bd2da324b87e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:08.598617 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea1da8a-39a0-4ff0-8603-bd2da324b87e None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:08.599090 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea1da8a-39a0-4ff0-8603-bd2da324b87e None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:08.599566 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea1da8a-39a0-4ff0-8603-bd2da324b87e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:08.653257 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:08.674828 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea1da8a-39a0-4ff0-8603-bd2da324b87e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:08.676699 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea1da8a-39a0-4ff0-8603-bd2da324b87e None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:08.825507 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 148/292] 10.222.0.27 () {58 vars in 1282 bytes} [Fri Aug 5 19:49:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 492 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 05 19:49:08.831229 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211a8f02-a01d-49ad-b747-11cf5b090213 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:08.969811 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:08.982289 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211a8f02-a01d-49ad-b747-11cf5b090213 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:09.049020 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8800254a-f475-428f-be47-512aff2bae54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:09.049816 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8800254a-f475-428f-be47-512aff2bae54 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:09.049976 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8800254a-f475-428f-be47-512aff2bae54 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:09.050252 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8800254a-f475-428f-be47-512aff2bae54 None admin] PATH_INFO: `/v3/projects/6440bf75a9134de7ba2c5fb9c23955e2` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:09.051456 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8800254a-f475-428f-be47-512aff2bae54 None admin] RBAC: Authorizing `identity:get_project(project_id=6440bf75a9134de7ba2c5fb9c23955e2)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:09.053265 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8800254a-f475-428f-be47-512aff2bae54 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:09.055421 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 145/293] 10.222.0.27 () {56 vars in 1186 bytes} [Fri Aug 5 19:49:08 2022] GET /identity/v3/projects/6440bf75a9134de7ba2c5fb9c23955e2 => generated 302 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:49:09.061627 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea1da8a-39a0-4ff0-8603-bd2da324b87e None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:09.201473 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:09.231217 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea1da8a-39a0-4ff0-8603-bd2da324b87e None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:09.283256 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac145137-f1da-429e-aa91-80aa42ba3f3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:09.284079 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac145137-f1da-429e-aa91-80aa42ba3f3b None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:09.284452 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac145137-f1da-429e-aa91-80aa42ba3f3b None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:09.284861 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac145137-f1da-429e-aa91-80aa42ba3f3b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:09.285887 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac145137-f1da-429e-aa91-80aa42ba3f3b None admin] RBAC: Adding query filter params (role.id=1a4f694d53054a24b580138b1ba77f3c, scope.project.id=6440bf75a9134de7ba2c5fb9c23955e2, user.id=9f8c69917bf44d1aaa26ac61b516a598) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:49:09.286196 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac145137-f1da-429e-aa91-80aa42ba3f3b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:09.287780 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac145137-f1da-429e-aa91-80aa42ba3f3b None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:09.298091 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 149/294] 10.222.0.27 () {56 vars in 1374 bytes} [Fri Aug 5 19:49:09 2022] GET /identity/v3/role_assignments?user.id=9f8c69917bf44d1aaa26ac61b516a598&scope.project.id=6440bf75a9134de7ba2c5fb9c23955e2&role.id=1a4f694d53054a24b580138b1ba77f3c => generated 608 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:49:09.797947 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e9355f-7cec-4d9d-bbc6-4a917250078f None None] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:09.798331 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e9355f-7cec-4d9d-bbc6-4a917250078f None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:09.798505 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e9355f-7cec-4d9d-bbc6-4a917250078f None None] PATH_INFO: `/` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:09.799891 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 146/295] 10.222.0.27 () {54 vars in 927 bytes} [Fri Aug 5 19:49:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:49:09.805420 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bee58c-ccb3-47b1-acae-e6dfaa4d9bdd None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:09.805999 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bee58c-ccb3-47b1-acae-e6dfaa4d9bdd None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:09.806471 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bee58c-ccb3-47b1-acae-e6dfaa4d9bdd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:09.826558 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-96bee58c-ccb3-47b1-acae-e6dfaa4d9bdd None None] MFA Rules not processed for user `03d1834a64cd4a43829827cb99a72747`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:49:09.828160 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-96bee58c-ccb3-47b1-acae-e6dfaa4d9bdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:49:09.904466 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 150/296] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Aug 5 19:49:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 19:49:09.928192 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e9355f-7cec-4d9d-bbc6-4a917250078f None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:10.069106 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e9355f-7cec-4d9d-bbc6-4a917250078f None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:10.104007 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a33abd-61e0-4384-9e3d-81b54119172e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:10.104695 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a33abd-61e0-4384-9e3d-81b54119172e None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:10.105048 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a33abd-61e0-4384-9e3d-81b54119172e None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:10.105297 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a33abd-61e0-4384-9e3d-81b54119172e None admin] PATH_INFO: `/v3/roles/member` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:10.117814 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a33abd-61e0-4384-9e3d-81b54119172e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 19:49:10.123054 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application [None req-43a33abd-61e0-4384-9e3d-81b54119172e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 05 19:49:10.123054 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:49:10.123054 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:49:10.123054 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:49:10.123054 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:49:10.123054 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:49:10.123054 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:49:10.123054 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:49:10.123054 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:49:10.123054 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:49:10.123054 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:49:10.123054 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:49:10.123054 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 19:49:10.123054 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 19:49:10.123054 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 19:49:10.123054 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 19:49:10.124886 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 19:49:10.124886 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 19:49:10.124886 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 19:49:10.124886 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 19:49:10.124886 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:49:10.124886 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:49:10.124886 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:49:10.124886 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:49:10.124886 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:49:10.124886 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:49:10.124886 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:49:10.124886 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application with Lock( Aug 05 19:49:10.124886 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:49:10.124886 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:49:10.124886 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:49:10.124886 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:49:10.126513 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:49:10.126513 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:49:10.126513 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:49:10.126513 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:49:10.126513 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 19:49:10.126513 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 19:49:10.126513 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 19:49:10.126513 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 19:49:10.126513 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 19:49:10.126513 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 19:49:10.126513 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 05 19:49:10.126513 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Aug 05 19:49:10.126513 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 147/297] 10.222.0.27 () {56 vars in 1099 bytes} [Fri Aug 5 19:49:09 2022] GET /identity/v3/roles/member => generated 84 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 05 19:49:10.138235 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96bee58c-ccb3-47b1-acae-e6dfaa4d9bdd None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:10.159734 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1be4d8b-1363-45bf-b830-3d31f4a99a1d None None] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:10.160125 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1be4d8b-1363-45bf-b830-3d31f4a99a1d None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:10.160298 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1be4d8b-1363-45bf-b830-3d31f4a99a1d None None] PATH_INFO: `/` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:10.161884 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 148/298] 10.222.0.27 () {54 vars in 927 bytes} [Fri Aug 5 19:49:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 05 19:49:10.167224 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af6b920-101a-4e14-bfb1-28d0d9f0d084 None None] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:10.167688 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af6b920-101a-4e14-bfb1-28d0d9f0d084 None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:10.168107 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af6b920-101a-4e14-bfb1-28d0d9f0d084 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:10.184862 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:10.187131 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.auth.core [None req-4af6b920-101a-4e14-bfb1-28d0d9f0d084 None None] MFA Rules not processed for user `03d1834a64cd4a43829827cb99a72747`. Rule list: `[]` (Enabled: `True`). {{(pid=80364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:49:10.188728 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-4af6b920-101a-4e14-bfb1-28d0d9f0d084 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:49:10.275951 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:10.278059 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 149/299] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Aug 5 19:49:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 05 19:49:10.298651 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af6b920-101a-4e14-bfb1-28d0d9f0d084 None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:10.304603 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96bee58c-ccb3-47b1-acae-e6dfaa4d9bdd None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:10.349933 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa8145a-c10e-49f9-8d87-b84485f61179 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:10.350577 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa8145a-c10e-49f9-8d87-b84485f61179 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:10.350869 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa8145a-c10e-49f9-8d87-b84485f61179 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:10.351171 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa8145a-c10e-49f9-8d87-b84485f61179 None admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:10.351957 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa8145a-c10e-49f9-8d87-b84485f61179 None admin] RBAC: Adding query filter params (name=member) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:49:10.352267 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa8145a-c10e-49f9-8d87-b84485f61179 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:10.353776 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa8145a-c10e-49f9-8d87-b84485f61179 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:10.374100 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 151/300] 10.222.0.27 () {56 vars in 1101 bytes} [Fri Aug 5 19:49:10 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:49:10.375715 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f0a49c-1924-4599-8611-5e864726872d None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:10.376151 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f0a49c-1924-4599-8611-5e864726872d None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:10.376593 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f0a49c-1924-4599-8611-5e864726872d None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:10.378545 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 152/301] 10.222.0.27 () {54 vars in 927 bytes} [Fri Aug 5 19:49:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:49:10.384831 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f0a49c-1924-4599-8611-5e864726872d None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:10.430689 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af6b920-101a-4e14-bfb1-28d0d9f0d084 None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:10.470670 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38f8ea6-4800-4856-afdb-9becbcf7dbf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:10.471577 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38f8ea6-4800-4856-afdb-9becbcf7dbf2 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:10.471853 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38f8ea6-4800-4856-afdb-9becbcf7dbf2 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:10.472277 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38f8ea6-4800-4856-afdb-9becbcf7dbf2 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:10.475151 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:10.484217 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c38f8ea6-4800-4856-afdb-9becbcf7dbf2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 19:49:10.489364 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application [None req-c38f8ea6-4800-4856-afdb-9becbcf7dbf2 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 05 19:49:10.489364 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:49:10.489364 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:49:10.489364 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:49:10.489364 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:49:10.489364 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:49:10.489364 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:49:10.489364 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:49:10.489364 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:49:10.489364 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:49:10.489364 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:49:10.489364 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:49:10.489364 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 19:49:10.489364 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 19:49:10.489364 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 19:49:10.489364 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 19:49:10.490448 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 19:49:10.490448 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 19:49:10.490448 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 19:49:10.490448 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 19:49:10.490448 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:49:10.490448 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:49:10.490448 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:49:10.490448 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:49:10.490448 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:49:10.490448 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:49:10.490448 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:49:10.490448 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application with Lock( Aug 05 19:49:10.490448 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:49:10.490448 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:49:10.490448 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:49:10.490448 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:49:10.491466 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:49:10.491466 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:49:10.491466 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:49:10.491466 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:49:10.491466 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 19:49:10.491466 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 19:49:10.491466 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 19:49:10.491466 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 19:49:10.491466 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 19:49:10.491466 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 19:49:10.491466 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 05 19:49:10.491466 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Aug 05 19:49:10.491466 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 150/302] 10.222.0.27 () {56 vars in 1099 bytes} [Fri Aug 5 19:49:10 2022] GET /identity/v3/roles/reader => generated 84 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:49:10.497121 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111ba578-fdc3-43de-98cb-d3cd6d8ffc11 None None] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:10.497403 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111ba578-fdc3-43de-98cb-d3cd6d8ffc11 None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:10.497705 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111ba578-fdc3-43de-98cb-d3cd6d8ffc11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:10.503440 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f0a49c-1924-4599-8611-5e864726872d None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:10.517565 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.auth.core [None req-111ba578-fdc3-43de-98cb-d3cd6d8ffc11 None None] MFA Rules not processed for user `03d1834a64cd4a43829827cb99a72747`. Rule list: `[]` (Enabled: `True`). {{(pid=80364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:49:10.518756 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-111ba578-fdc3-43de-98cb-d3cd6d8ffc11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:49:10.543989 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbee9f1d-99a9-42e2-bab4-c31d49d41a16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:10.544682 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbee9f1d-99a9-42e2-bab4-c31d49d41a16 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:10.545209 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbee9f1d-99a9-42e2-bab4-c31d49d41a16 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:10.545209 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbee9f1d-99a9-42e2-bab4-c31d49d41a16 None admin] PATH_INFO: `/v3/users/d43d81977c134189a3fbfc92c8e6b52e` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:10.546406 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbee9f1d-99a9-42e2-bab4-c31d49d41a16 None admin] RBAC: Authorizing `identity:get_user(user_id=d43d81977c134189a3fbfc92c8e6b52e)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:10.547898 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbee9f1d-99a9-42e2-bab4-c31d49d41a16 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:10.554126 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 153/303] 10.222.0.27 () {56 vars in 1177 bytes} [Fri Aug 5 19:49:10 2022] GET /identity/v3/users/d43d81977c134189a3fbfc92c8e6b52e => generated 293 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:49:10.554126 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbee9f1d-99a9-42e2-bab4-c31d49d41a16 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:10.559713 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:10.600487 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 151/304] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Aug 5 19:49:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 19:49:10.601256 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111ba578-fdc3-43de-98cb-d3cd6d8ffc11 None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:10.673367 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbee9f1d-99a9-42e2-bab4-c31d49d41a16 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:10.712904 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0efcefb-22de-4c8f-b846-5dece84d1f67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:10.713526 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0efcefb-22de-4c8f-b846-5dece84d1f67 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:10.713800 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0efcefb-22de-4c8f-b846-5dece84d1f67 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:10.714069 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0efcefb-22de-4c8f-b846-5dece84d1f67 None admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:10.714862 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0efcefb-22de-4c8f-b846-5dece84d1f67 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:49:10.715094 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0efcefb-22de-4c8f-b846-5dece84d1f67 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:10.716450 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0efcefb-22de-4c8f-b846-5dece84d1f67 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:10.721828 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111ba578-fdc3-43de-98cb-d3cd6d8ffc11 None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:10.726172 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 154/305] 10.222.0.27 () {56 vars in 1101 bytes} [Fri Aug 5 19:49:10 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:49:10.726838 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0efcefb-22de-4c8f-b846-5dece84d1f67 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:10.752278 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:10.762215 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423c411e-b833-4183-84db-297876847395 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:10.762879 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423c411e-b833-4183-84db-297876847395 None admin] REQUEST_METHOD: `PUT` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:10.763160 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423c411e-b833-4183-84db-297876847395 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:10.763435 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423c411e-b833-4183-84db-297876847395 None admin] PATH_INFO: `/v3/system/users/d43d81977c134189a3fbfc92c8e6b52e/roles/1a4f694d53054a24b580138b1ba77f3c` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:10.764927 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-423c411e-b833-4183-84db-297876847395 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=d43d81977c134189a3fbfc92c8e6b52e, role_id=1a4f694d53054a24b580138b1ba77f3c)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:10.766933 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-423c411e-b833-4183-84db-297876847395 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:10.784538 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 152/306] 10.222.0.27 () {58 vars in 1334 bytes} [Fri Aug 5 19:49:10 2022] PUT /identity/v3/system/users/d43d81977c134189a3fbfc92c8e6b52e/roles/1a4f694d53054a24b580138b1ba77f3c => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 19:49:10.784538 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423c411e-b833-4183-84db-297876847395 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:10.819417 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:10.864318 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0efcefb-22de-4c8f-b846-5dece84d1f67 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:10.914705 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6e9b96-2c06-412e-874d-583ad4afc327 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:10.915955 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6e9b96-2c06-412e-874d-583ad4afc327 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:10.916240 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6e9b96-2c06-412e-874d-583ad4afc327 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:10.916580 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6e9b96-2c06-412e-874d-583ad4afc327 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:10.918896 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6e9b96-2c06-412e-874d-583ad4afc327 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:10.920718 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6e9b96-2c06-412e-874d-583ad4afc327 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:10.924218 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 155/307] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Aug 5 19:49:10 2022] GET /identity/v3/domains/default => generated 199 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:49:10.930896 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6e9b96-2c06-412e-874d-583ad4afc327 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:10.950349 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423c411e-b833-4183-84db-297876847395 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:10.998072 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e037e0-8811-457f-a93e-60cccc43b013 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:10.998072 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e037e0-8811-457f-a93e-60cccc43b013 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:10.998072 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e037e0-8811-457f-a93e-60cccc43b013 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:10.998072 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e037e0-8811-457f-a93e-60cccc43b013 None admin] PATH_INFO: `/v3/users/bd35b64e464a49ce9a54e6a4723941c7` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:11.019265 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e037e0-8811-457f-a93e-60cccc43b013 None admin] RBAC: Authorizing `identity:get_user(user_id=bd35b64e464a49ce9a54e6a4723941c7)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:11.020792 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e037e0-8811-457f-a93e-60cccc43b013 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:11.022829 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 153/308] 10.222.0.27 () {56 vars in 1177 bytes} [Fri Aug 5 19:49:10 2022] GET /identity/v3/users/bd35b64e464a49ce9a54e6a4723941c7 => generated 293 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:49:11.028951 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e037e0-8811-457f-a93e-60cccc43b013 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:11.079038 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6e9b96-2c06-412e-874d-583ad4afc327 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:11.086532 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:11.113302 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1aa30d8-6d58-4b67-a298-ed6c5c848937 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:11.113920 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1aa30d8-6d58-4b67-a298-ed6c5c848937 None admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:11.114197 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1aa30d8-6d58-4b67-a298-ed6c5c848937 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:11.114520 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1aa30d8-6d58-4b67-a298-ed6c5c848937 None admin] PATH_INFO: `/v3/groups` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:11.115127 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1aa30d8-6d58-4b67-a298-ed6c5c848937 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:11.116591 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1aa30d8-6d58-4b67-a298-ed6c5c848937 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:11.122839 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:11.136300 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 156/309] 10.222.0.27 () {60 vars in 1134 bytes} [Fri Aug 5 19:49:10 2022] POST /identity/v3/groups => generated 229 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 19:49:11.137510 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9ca6cb-5abc-4610-a7e8-ef243d72c67b None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:11.137795 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9ca6cb-5abc-4610-a7e8-ef243d72c67b None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:11.138068 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9ca6cb-5abc-4610-a7e8-ef243d72c67b None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:11.139302 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 157/310] 10.222.0.27 () {54 vars in 927 bytes} [Fri Aug 5 19:49:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:49:11.145089 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83abf799-50d4-4a0f-a35c-31495a9391c9 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:11.145374 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83abf799-50d4-4a0f-a35c-31495a9391c9 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:11.145654 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83abf799-50d4-4a0f-a35c-31495a9391c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:11.161137 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e037e0-8811-457f-a93e-60cccc43b013 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:11.166393 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-83abf799-50d4-4a0f-a35c-31495a9391c9 None None] MFA Rules not processed for user `03d1834a64cd4a43829827cb99a72747`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:49:11.167748 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-83abf799-50d4-4a0f-a35c-31495a9391c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:49:11.217102 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbecea8b-358e-4506-bdb5-f2d8ac99e32d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:11.219654 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbecea8b-358e-4506-bdb5-f2d8ac99e32d None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:11.220045 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbecea8b-358e-4506-bdb5-f2d8ac99e32d None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:11.220469 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbecea8b-358e-4506-bdb5-f2d8ac99e32d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:11.221492 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbecea8b-358e-4506-bdb5-f2d8ac99e32d None admin] RBAC: Adding query filter params (role.id=b885e5d9af42483799785b59b26b6207, user.id=bd35b64e464a49ce9a54e6a4723941c7, scope.system=all) {{(pid=80364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:49:11.221946 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbecea8b-358e-4506-bdb5-f2d8ac99e32d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:11.224550 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbecea8b-358e-4506-bdb5-f2d8ac99e32d None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:11.236978 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 154/311] 10.222.0.27 () {56 vars in 1308 bytes} [Fri Aug 5 19:49:11 2022] GET /identity/v3/role_assignments?user.id=bd35b64e464a49ce9a54e6a4723941c7&scope.system=all&role.id=b885e5d9af42483799785b59b26b6207 => generated 226 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:49:11.264060 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 158/312] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Aug 5 19:49:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 05 19:49:11.288877 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbecea8b-358e-4506-bdb5-f2d8ac99e32d None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:11.407965 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbecea8b-358e-4506-bdb5-f2d8ac99e32d None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:11.423106 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:11.445294 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744d60f1-e0bd-4a89-88a9-d579b5476609 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:11.445936 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744d60f1-e0bd-4a89-88a9-d579b5476609 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:11.446218 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744d60f1-e0bd-4a89-88a9-d579b5476609 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:11.446485 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744d60f1-e0bd-4a89-88a9-d579b5476609 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:11.457419 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-744d60f1-e0bd-4a89-88a9-d579b5476609 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 19:49:11.462285 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application [None req-744d60f1-e0bd-4a89-88a9-d579b5476609 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 05 19:49:11.462285 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:49:11.462285 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:49:11.462285 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:49:11.462285 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:49:11.462285 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:49:11.462285 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:49:11.462285 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:49:11.462285 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:49:11.462285 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:49:11.462285 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:49:11.462285 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:49:11.462285 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 19:49:11.462285 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 19:49:11.462285 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 19:49:11.462285 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 19:49:11.463264 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 19:49:11.463264 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 19:49:11.463264 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 19:49:11.463264 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 19:49:11.463264 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:49:11.463264 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:49:11.463264 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:49:11.463264 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:49:11.463264 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:49:11.463264 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:49:11.463264 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:49:11.463264 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application with Lock( Aug 05 19:49:11.463264 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:49:11.463264 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:49:11.463264 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:49:11.463264 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:49:11.464055 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:49:11.464055 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:49:11.464055 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:49:11.464055 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:49:11.464055 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 19:49:11.464055 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 19:49:11.464055 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 19:49:11.464055 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 19:49:11.464055 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 19:49:11.464055 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 19:49:11.464055 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 05 19:49:11.464055 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Aug 05 19:49:11.464055 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 155/313] 10.222.0.27 () {56 vars in 1099 bytes} [Fri Aug 5 19:49:11 2022] GET /identity/v3/roles/reader => generated 84 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:49:11.469888 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83abf799-50d4-4a0f-a35c-31495a9391c9 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:11.595637 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83abf799-50d4-4a0f-a35c-31495a9391c9 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:11.624543 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:11.634174 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752f1843-4127-4969-a434-0c1920faaf15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:11.634808 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752f1843-4127-4969-a434-0c1920faaf15 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:11.635090 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752f1843-4127-4969-a434-0c1920faaf15 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:11.635365 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752f1843-4127-4969-a434-0c1920faaf15 None admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:11.636102 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-752f1843-4127-4969-a434-0c1920faaf15 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:49:11.636409 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-752f1843-4127-4969-a434-0c1920faaf15 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:11.637863 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-752f1843-4127-4969-a434-0c1920faaf15 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:11.652395 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 159/314] 10.222.0.27 () {56 vars in 1101 bytes} [Fri Aug 5 19:49:11 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:49:11.655478 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744d60f1-e0bd-4a89-88a9-d579b5476609 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:11.782752 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744d60f1-e0bd-4a89-88a9-d579b5476609 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:11.818891 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2116ac-79a3-48d6-b86e-bbb822627502 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:11.819935 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2116ac-79a3-48d6-b86e-bbb822627502 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:11.820420 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2116ac-79a3-48d6-b86e-bbb822627502 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:11.820932 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2116ac-79a3-48d6-b86e-bbb822627502 None admin] PATH_INFO: `/v3/users/c5c7582de1584235b7891d319ef5761a` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:11.822435 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb2116ac-79a3-48d6-b86e-bbb822627502 None admin] RBAC: Authorizing `identity:get_user(user_id=c5c7582de1584235b7891d319ef5761a)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:11.824168 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb2116ac-79a3-48d6-b86e-bbb822627502 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:11.826573 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 156/315] 10.222.0.27 () {56 vars in 1177 bytes} [Fri Aug 5 19:49:11 2022] GET /identity/v3/users/c5c7582de1584235b7891d319ef5761a => generated 297 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 19:49:11.832445 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752f1843-4127-4969-a434-0c1920faaf15 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:11.939843 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752f1843-4127-4969-a434-0c1920faaf15 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:11.980762 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19735a2c-55d6-47ae-9178-2a48b4f93308 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:11.981409 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19735a2c-55d6-47ae-9178-2a48b4f93308 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:11.981702 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19735a2c-55d6-47ae-9178-2a48b4f93308 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:11.981961 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19735a2c-55d6-47ae-9178-2a48b4f93308 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:12.049830 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:12.060919 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19735a2c-55d6-47ae-9178-2a48b4f93308 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:12.063551 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19735a2c-55d6-47ae-9178-2a48b4f93308 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:12.167081 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 160/316] 10.222.0.27 () {58 vars in 1282 bytes} [Fri Aug 5 19:49:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 05 19:49:12.172831 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2116ac-79a3-48d6-b86e-bbb822627502 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:12.217260 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:12.284860 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2116ac-79a3-48d6-b86e-bbb822627502 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:12.322213 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde340da-1435-4bd4-b5a8-d8f7d1cbb592 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:12.322874 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde340da-1435-4bd4-b5a8-d8f7d1cbb592 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:12.323218 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde340da-1435-4bd4-b5a8-d8f7d1cbb592 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:12.323460 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde340da-1435-4bd4-b5a8-d8f7d1cbb592 None admin] PATH_INFO: `/v3/projects/6440bf75a9134de7ba2c5fb9c23955e2` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:12.324730 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde340da-1435-4bd4-b5a8-d8f7d1cbb592 None admin] RBAC: Authorizing `identity:get_project(project_id=6440bf75a9134de7ba2c5fb9c23955e2)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:12.326181 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde340da-1435-4bd4-b5a8-d8f7d1cbb592 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:12.328712 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 157/317] 10.222.0.27 () {56 vars in 1186 bytes} [Fri Aug 5 19:49:12 2022] GET /identity/v3/projects/6440bf75a9134de7ba2c5fb9c23955e2 => generated 302 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:49:12.334616 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19735a2c-55d6-47ae-9178-2a48b4f93308 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:12.431541 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:12.445499 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19735a2c-55d6-47ae-9178-2a48b4f93308 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:12.482889 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f973c6-7b2f-406c-baad-44bc8d83b0fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:12.483437 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f973c6-7b2f-406c-baad-44bc8d83b0fd None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:12.483718 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f973c6-7b2f-406c-baad-44bc8d83b0fd None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:12.483992 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f973c6-7b2f-406c-baad-44bc8d83b0fd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:12.485367 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f973c6-7b2f-406c-baad-44bc8d83b0fd None admin] RBAC: Adding query filter params (role.id=b885e5d9af42483799785b59b26b6207, scope.project.id=6440bf75a9134de7ba2c5fb9c23955e2, user.id=c5c7582de1584235b7891d319ef5761a) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:49:12.485746 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f973c6-7b2f-406c-baad-44bc8d83b0fd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:12.487212 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f973c6-7b2f-406c-baad-44bc8d83b0fd None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:12.494150 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 161/318] 10.222.0.27 () {56 vars in 1374 bytes} [Fri Aug 5 19:49:12 2022] GET /identity/v3/role_assignments?user.id=c5c7582de1584235b7891d319ef5761a&scope.project.id=6440bf75a9134de7ba2c5fb9c23955e2&role.id=b885e5d9af42483799785b59b26b6207 => generated 608 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:49:12.939351 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d998df1c-71d3-47d5-9c54-66aef09d692f None None] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:12.939747 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d998df1c-71d3-47d5-9c54-66aef09d692f None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:12.939940 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d998df1c-71d3-47d5-9c54-66aef09d692f None None] PATH_INFO: `/` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:12.941585 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 158/319] 10.222.0.27 () {54 vars in 927 bytes} [Fri Aug 5 19:49:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:49:12.946849 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0638512-ff73-4b43-8e61-fcda5b12b4ce None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:12.947136 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0638512-ff73-4b43-8e61-fcda5b12b4ce None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:12.947407 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0638512-ff73-4b43-8e61-fcda5b12b4ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:12.966788 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-c0638512-ff73-4b43-8e61-fcda5b12b4ce None None] MFA Rules not processed for user `03d1834a64cd4a43829827cb99a72747`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:49:12.967970 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-c0638512-ff73-4b43-8e61-fcda5b12b4ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:49:13.052889 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 162/320] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Aug 5 19:49:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 19:49:13.077493 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d998df1c-71d3-47d5-9c54-66aef09d692f None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:13.182674 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d998df1c-71d3-47d5-9c54-66aef09d692f None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:13.193105 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:13.219787 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70cca099-4e4a-4d9d-b6df-bd571a0342ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:13.220157 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cca099-4e4a-4d9d-b6df-bd571a0342ee None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:13.220396 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cca099-4e4a-4d9d-b6df-bd571a0342ee None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:13.220706 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cca099-4e4a-4d9d-b6df-bd571a0342ee None admin] PATH_INFO: `/v3/roles/member` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:13.231165 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70cca099-4e4a-4d9d-b6df-bd571a0342ee None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 19:49:13.236693 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application [None req-70cca099-4e4a-4d9d-b6df-bd571a0342ee None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 05 19:49:13.236693 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:49:13.236693 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:49:13.236693 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:49:13.236693 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:49:13.236693 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:49:13.236693 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:49:13.236693 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:49:13.236693 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:49:13.236693 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:49:13.236693 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:49:13.236693 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:49:13.236693 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 19:49:13.236693 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 19:49:13.236693 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 19:49:13.236693 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 19:49:13.237584 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 19:49:13.237584 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 19:49:13.237584 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 19:49:13.237584 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 19:49:13.237584 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:49:13.237584 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:49:13.237584 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:49:13.237584 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:49:13.237584 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:49:13.237584 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:49:13.237584 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:49:13.237584 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application with Lock( Aug 05 19:49:13.237584 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:49:13.237584 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:49:13.237584 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:49:13.237584 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:49:13.238378 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:49:13.238378 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:49:13.238378 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:49:13.238378 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:49:13.238378 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 19:49:13.238378 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 19:49:13.238378 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 19:49:13.238378 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 19:49:13.238378 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 19:49:13.238378 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 19:49:13.238378 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 05 19:49:13.238378 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Aug 05 19:49:13.238378 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 159/321] 10.222.0.27 () {56 vars in 1099 bytes} [Fri Aug 5 19:49:13 2022] GET /identity/v3/roles/member => generated 84 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 05 19:49:13.245225 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0638512-ff73-4b43-8e61-fcda5b12b4ce None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:13.306464 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b06dacb-cc57-4379-823e-32d9a487c7ca None None] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:13.306750 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b06dacb-cc57-4379-823e-32d9a487c7ca None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:13.307026 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b06dacb-cc57-4379-823e-32d9a487c7ca None None] PATH_INFO: `/` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:13.308550 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 160/322] 10.222.0.27 () {54 vars in 927 bytes} [Fri Aug 5 19:49:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 05 19:49:13.314332 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ae3077-2523-42e8-bd52-5a8f18825775 None None] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:13.314619 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ae3077-2523-42e8-bd52-5a8f18825775 None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:13.314893 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ae3077-2523-42e8-bd52-5a8f18825775 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:13.333713 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.auth.core [None req-93ae3077-2523-42e8-bd52-5a8f18825775 None None] MFA Rules not processed for user `03d1834a64cd4a43829827cb99a72747`. Rule list: `[]` (Enabled: `True`). {{(pid=80364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:49:13.334912 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:13.335024 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-93ae3077-2523-42e8-bd52-5a8f18825775 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:49:13.360196 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0638512-ff73-4b43-8e61-fcda5b12b4ce None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:13.405387 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c17347-7b6b-4a86-96fe-af3dbf8142ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:13.406550 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c17347-7b6b-4a86-96fe-af3dbf8142ba None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:13.406817 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c17347-7b6b-4a86-96fe-af3dbf8142ba None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:13.407090 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c17347-7b6b-4a86-96fe-af3dbf8142ba None admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:13.407813 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c17347-7b6b-4a86-96fe-af3dbf8142ba None admin] RBAC: Adding query filter params (name=member) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:49:13.408124 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c17347-7b6b-4a86-96fe-af3dbf8142ba None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:13.409548 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c17347-7b6b-4a86-96fe-af3dbf8142ba None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:13.418485 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 163/323] 10.222.0.27 () {56 vars in 1101 bytes} [Fri Aug 5 19:49:13 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:49:13.424713 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c17347-7b6b-4a86-96fe-af3dbf8142ba None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:13.426908 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 161/324] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Aug 5 19:49:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 05 19:49:13.448727 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d7f373-1de4-438c-9bb4-a11105b95f93 None None] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:13.448912 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d7f373-1de4-438c-9bb4-a11105b95f93 None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:13.449184 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d7f373-1de4-438c-9bb4-a11105b95f93 None None] PATH_INFO: `/` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:13.450557 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 162/325] 10.222.0.27 () {54 vars in 927 bytes} [Fri Aug 5 19:49:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 05 19:49:13.453314 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d7f373-1de4-438c-9bb4-a11105b95f93 None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:13.527448 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c17347-7b6b-4a86-96fe-af3dbf8142ba None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:13.566786 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584acffe-bff2-4323-8b30-4f7ae0bac2c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:13.567655 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584acffe-bff2-4323-8b30-4f7ae0bac2c6 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:13.568186 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584acffe-bff2-4323-8b30-4f7ae0bac2c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:13.568741 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584acffe-bff2-4323-8b30-4f7ae0bac2c6 None admin] PATH_INFO: `/v3/users/d43d81977c134189a3fbfc92c8e6b52e` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:13.569971 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:13.570177 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-584acffe-bff2-4323-8b30-4f7ae0bac2c6 None admin] RBAC: Authorizing `identity:get_user(user_id=d43d81977c134189a3fbfc92c8e6b52e)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:13.571948 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-584acffe-bff2-4323-8b30-4f7ae0bac2c6 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:13.574078 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 164/326] 10.222.0.27 () {56 vars in 1177 bytes} [Fri Aug 5 19:49:13 2022] GET /identity/v3/users/d43d81977c134189a3fbfc92c8e6b52e => generated 293 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 19:49:13.576007 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd0edab-4fcd-4900-800d-c15b78510273 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:13.576296 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd0edab-4fcd-4900-800d-c15b78510273 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:13.576603 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd0edab-4fcd-4900-800d-c15b78510273 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:13.580841 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d7f373-1de4-438c-9bb4-a11105b95f93 None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:13.598262 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-4bd0edab-4fcd-4900-800d-c15b78510273 None None] MFA Rules not processed for user `03d1834a64cd4a43829827cb99a72747`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:49:13.599470 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-4bd0edab-4fcd-4900-800d-c15b78510273 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:49:13.607117 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:13.620739 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88d1615-de94-4a25-b124-38df87c9bf21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:13.621377 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88d1615-de94-4a25-b124-38df87c9bf21 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:13.621657 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88d1615-de94-4a25-b124-38df87c9bf21 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:13.621926 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88d1615-de94-4a25-b124-38df87c9bf21 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:13.631944 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e88d1615-de94-4a25-b124-38df87c9bf21 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 19:49:13.637931 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application [None req-e88d1615-de94-4a25-b124-38df87c9bf21 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 05 19:49:13.637931 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:49:13.637931 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:49:13.637931 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:49:13.637931 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:49:13.637931 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:49:13.637931 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:49:13.637931 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:49:13.637931 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:49:13.637931 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:49:13.637931 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:49:13.637931 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:49:13.637931 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 19:49:13.637931 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 19:49:13.637931 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 19:49:13.637931 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 19:49:13.638874 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 19:49:13.638874 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 19:49:13.638874 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 19:49:13.638874 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 19:49:13.638874 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:49:13.638874 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:49:13.638874 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:49:13.638874 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:49:13.638874 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:49:13.638874 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:49:13.638874 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:49:13.638874 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application with Lock( Aug 05 19:49:13.638874 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:49:13.638874 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:49:13.638874 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:49:13.638874 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:49:13.639676 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:49:13.639676 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:49:13.639676 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:49:13.639676 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:49:13.639676 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 19:49:13.639676 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 19:49:13.639676 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 19:49:13.639676 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 19:49:13.639676 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 19:49:13.639676 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 19:49:13.639676 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 05 19:49:13.639676 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Aug 05 19:49:13.639676 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 163/327] 10.222.0.27 () {56 vars in 1099 bytes} [Fri Aug 5 19:49:13 2022] GET /identity/v3/roles/reader => generated 84 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:49:13.644862 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88d1615-de94-4a25-b124-38df87c9bf21 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:13.670166 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 165/328] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Aug 5 19:49:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 19:49:13.670166 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd0edab-4fcd-4900-800d-c15b78510273 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:13.747668 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88d1615-de94-4a25-b124-38df87c9bf21 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:13.779032 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd0edab-4fcd-4900-800d-c15b78510273 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:13.782001 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977db6fb-d621-4b2e-ac37-ab1737afafff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:13.782678 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977db6fb-d621-4b2e-ac37-ab1737afafff None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:13.782986 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977db6fb-d621-4b2e-ac37-ab1737afafff None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:13.783222 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977db6fb-d621-4b2e-ac37-ab1737afafff None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:13.784005 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977db6fb-d621-4b2e-ac37-ab1737afafff None admin] RBAC: Adding query filter params (role.id=1a4f694d53054a24b580138b1ba77f3c, user.id=d43d81977c134189a3fbfc92c8e6b52e, scope.system=all) {{(pid=80364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:49:13.784309 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977db6fb-d621-4b2e-ac37-ab1737afafff None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:13.785792 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977db6fb-d621-4b2e-ac37-ab1737afafff None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:13.797280 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 164/329] 10.222.0.27 () {56 vars in 1308 bytes} [Fri Aug 5 19:49:13 2022] GET /identity/v3/role_assignments?user.id=d43d81977c134189a3fbfc92c8e6b52e&scope.system=all&role.id=1a4f694d53054a24b580138b1ba77f3c => generated 510 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:49:13.798087 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977db6fb-d621-4b2e-ac37-ab1737afafff None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:13.828779 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:13.831919 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158b25ce-cea3-4b6e-b3a6-4706f493486d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:13.832591 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158b25ce-cea3-4b6e-b3a6-4706f493486d None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:13.832868 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158b25ce-cea3-4b6e-b3a6-4706f493486d None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:13.833139 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158b25ce-cea3-4b6e-b3a6-4706f493486d None admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:13.833876 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-158b25ce-cea3-4b6e-b3a6-4706f493486d None admin] RBAC: Adding query filter params (name=reader) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:49:13.834177 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-158b25ce-cea3-4b6e-b3a6-4706f493486d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:13.835594 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-158b25ce-cea3-4b6e-b3a6-4706f493486d None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:13.844919 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 166/330] 10.222.0.27 () {56 vars in 1101 bytes} [Fri Aug 5 19:49:13 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:49:13.850550 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158b25ce-cea3-4b6e-b3a6-4706f493486d None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:13.854861 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:13.918546 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977db6fb-d621-4b2e-ac37-ab1737afafff None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:13.957609 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da570d08-b8a4-47c0-8db6-63a9d7538903 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:13.958270 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da570d08-b8a4-47c0-8db6-63a9d7538903 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:13.958556 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da570d08-b8a4-47c0-8db6-63a9d7538903 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:13.958834 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da570d08-b8a4-47c0-8db6-63a9d7538903 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:13.961169 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da570d08-b8a4-47c0-8db6-63a9d7538903 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:13.962381 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da570d08-b8a4-47c0-8db6-63a9d7538903 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:13.964361 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 165/331] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Aug 5 19:49:13 2022] GET /identity/v3/domains/default => generated 199 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:49:13.970651 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da570d08-b8a4-47c0-8db6-63a9d7538903 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:13.974866 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158b25ce-cea3-4b6e-b3a6-4706f493486d None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:14.015946 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80cb1811-9c28-471c-8c7b-2bcafa5b61d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:14.016841 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cb1811-9c28-471c-8c7b-2bcafa5b61d4 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:14.016841 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cb1811-9c28-471c-8c7b-2bcafa5b61d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:14.016841 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cb1811-9c28-471c-8c7b-2bcafa5b61d4 None admin] PATH_INFO: `/v3/users/bd35b64e464a49ce9a54e6a4723941c7` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:14.019969 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80cb1811-9c28-471c-8c7b-2bcafa5b61d4 None admin] RBAC: Authorizing `identity:get_user(user_id=bd35b64e464a49ce9a54e6a4723941c7)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:14.021908 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80cb1811-9c28-471c-8c7b-2bcafa5b61d4 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:14.023567 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 167/332] 10.222.0.27 () {56 vars in 1177 bytes} [Fri Aug 5 19:49:13 2022] GET /identity/v3/users/bd35b64e464a49ce9a54e6a4723941c7 => generated 293 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 19:49:14.030004 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80cb1811-9c28-471c-8c7b-2bcafa5b61d4 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:14.065999 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:14.081857 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da570d08-b8a4-47c0-8db6-63a9d7538903 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:14.098625 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:14.116020 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84060b2-e19d-42b1-8c64-f6ed88c806c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:14.116673 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84060b2-e19d-42b1-8c64-f6ed88c806c2 None admin] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:14.116957 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84060b2-e19d-42b1-8c64-f6ed88c806c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:14.117235 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84060b2-e19d-42b1-8c64-f6ed88c806c2 None admin] PATH_INFO: `/v3/groups` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:14.117910 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a84060b2-e19d-42b1-8c64-f6ed88c806c2 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:14.119380 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a84060b2-e19d-42b1-8c64-f6ed88c806c2 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:14.131127 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80cb1811-9c28-471c-8c7b-2bcafa5b61d4 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:14.136606 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 166/333] 10.222.0.27 () {60 vars in 1134 bytes} [Fri Aug 5 19:49:13 2022] POST /identity/v3/groups => generated 226 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 05 19:49:14.169152 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0adc4257-9557-4cac-a4f6-48c065fd4251 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:14.169844 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0adc4257-9557-4cac-a4f6-48c065fd4251 None admin] REQUEST_METHOD: `PUT` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:14.170124 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0adc4257-9557-4cac-a4f6-48c065fd4251 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:14.170398 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0adc4257-9557-4cac-a4f6-48c065fd4251 None admin] PATH_INFO: `/v3/system/users/bd35b64e464a49ce9a54e6a4723941c7/roles/b885e5d9af42483799785b59b26b6207` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:14.172597 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0adc4257-9557-4cac-a4f6-48c065fd4251 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=bd35b64e464a49ce9a54e6a4723941c7, role_id=b885e5d9af42483799785b59b26b6207)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:14.174039 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0adc4257-9557-4cac-a4f6-48c065fd4251 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:14.183525 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 168/334] 10.222.0.27 () {58 vars in 1334 bytes} [Fri Aug 5 19:49:14 2022] PUT /identity/v3/system/users/bd35b64e464a49ce9a54e6a4723941c7/roles/b885e5d9af42483799785b59b26b6207 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 05 19:49:16.426370 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1cb923b-8de0-42ed-a748-69b9cecb46b8 None None] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:16.427369 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1cb923b-8de0-42ed-a748-69b9cecb46b8 None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:16.427859 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1cb923b-8de0-42ed-a748-69b9cecb46b8 None None] PATH_INFO: `/` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:16.429870 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 167/335] 10.222.0.27 () {54 vars in 927 bytes} [Fri Aug 5 19:49:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:49:16.435790 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce271cb-ce11-4e85-b964-0b1a24cdec32 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:16.436294 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce271cb-ce11-4e85-b964-0b1a24cdec32 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:16.436386 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce271cb-ce11-4e85-b964-0b1a24cdec32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:16.465844 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-5ce271cb-ce11-4e85-b964-0b1a24cdec32 None None] MFA Rules not processed for user `03d1834a64cd4a43829827cb99a72747`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:49:16.467044 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-5ce271cb-ce11-4e85-b964-0b1a24cdec32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:49:16.549899 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 169/336] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Aug 5 19:49:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 19:49:16.586146 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1cb923b-8de0-42ed-a748-69b9cecb46b8 None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:16.734617 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368cb240-23d8-4f08-bc27-e54816407f6c None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:16.735245 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368cb240-23d8-4f08-bc27-e54816407f6c None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:16.735732 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368cb240-23d8-4f08-bc27-e54816407f6c None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:16.740593 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1cb923b-8de0-42ed-a748-69b9cecb46b8 None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:16.742112 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 170/337] 10.222.0.27 () {54 vars in 927 bytes} [Fri Aug 5 19:49:16 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:49:16.749361 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb58948-6f5f-44e2-90b0-1de1dae8f468 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:16.750467 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb58948-6f5f-44e2-90b0-1de1dae8f468 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:16.750978 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb58948-6f5f-44e2-90b0-1de1dae8f468 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:16.789527 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-5cb58948-6f5f-44e2-90b0-1de1dae8f468 None None] MFA Rules not processed for user `03d1834a64cd4a43829827cb99a72747`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:49:16.791236 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-5cb58948-6f5f-44e2-90b0-1de1dae8f468 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:49:16.794180 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fffdfe2-a6dc-46fe-a775-2606f61f21ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:16.794791 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fffdfe2-a6dc-46fe-a775-2606f61f21ba None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:16.795136 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fffdfe2-a6dc-46fe-a775-2606f61f21ba None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:16.795411 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fffdfe2-a6dc-46fe-a775-2606f61f21ba None admin] PATH_INFO: `/v3/roles/member` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:16.816040 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fffdfe2-a6dc-46fe-a775-2606f61f21ba None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 19:49:16.821359 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application [None req-5fffdfe2-a6dc-46fe-a775-2606f61f21ba None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 05 19:49:16.821359 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:49:16.821359 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:49:16.821359 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:49:16.821359 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:49:16.821359 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:49:16.821359 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:49:16.821359 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:49:16.821359 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:49:16.821359 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:49:16.821359 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:49:16.821359 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:49:16.821359 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 19:49:16.821359 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 19:49:16.821359 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 19:49:16.821359 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 19:49:16.822397 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 19:49:16.822397 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 19:49:16.822397 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 19:49:16.822397 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 19:49:16.822397 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:49:16.822397 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:49:16.822397 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:49:16.822397 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:49:16.822397 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:49:16.822397 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:49:16.822397 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:49:16.822397 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application with Lock( Aug 05 19:49:16.822397 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:49:16.822397 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:49:16.822397 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:49:16.822397 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:49:16.823211 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:49:16.823211 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:49:16.823211 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:49:16.823211 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:49:16.823211 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 19:49:16.823211 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 19:49:16.823211 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 19:49:16.823211 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 19:49:16.823211 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 19:49:16.823211 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 19:49:16.823211 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 05 19:49:16.823211 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Aug 05 19:49:16.831335 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 168/338] 10.222.0.27 () {56 vars in 1099 bytes} [Fri Aug 5 19:49:16 2022] GET /identity/v3/roles/member => generated 84 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:49:16.832686 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fffdfe2-a6dc-46fe-a775-2606f61f21ba None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:16.843022 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:16.946680 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:16.962876 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 171/339] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Aug 5 19:49:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 05 19:49:17.007860 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fffdfe2-a6dc-46fe-a775-2606f61f21ba None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:17.016943 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb58948-6f5f-44e2-90b0-1de1dae8f468 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:17.069238 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce334104-adf3-4c81-9371-b0a662663515 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:17.069862 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce334104-adf3-4c81-9371-b0a662663515 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:17.070138 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce334104-adf3-4c81-9371-b0a662663515 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:17.070406 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce334104-adf3-4c81-9371-b0a662663515 None admin] PATH_INFO: `/v3/roles` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:17.071123 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce334104-adf3-4c81-9371-b0a662663515 None admin] RBAC: Adding query filter params (name=member) {{(pid=80364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:49:17.071424 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce334104-adf3-4c81-9371-b0a662663515 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:17.072841 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce334104-adf3-4c81-9371-b0a662663515 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:17.091351 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 169/340] 10.222.0.27 () {56 vars in 1101 bytes} [Fri Aug 5 19:49:16 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:49:17.116340 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce334104-adf3-4c81-9371-b0a662663515 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:17.201619 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb58948-6f5f-44e2-90b0-1de1dae8f468 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:17.234307 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:17.248058 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7566cc3-2c9a-4651-a84b-9864bc993d52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:17.249121 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7566cc3-2c9a-4651-a84b-9864bc993d52 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:17.249580 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7566cc3-2c9a-4651-a84b-9864bc993d52 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:17.250546 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7566cc3-2c9a-4651-a84b-9864bc993d52 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:17.267368 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7566cc3-2c9a-4651-a84b-9864bc993d52 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 19:49:17.272873 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-f7566cc3-2c9a-4651-a84b-9864bc993d52 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 05 19:49:17.272873 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:49:17.272873 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:49:17.272873 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:49:17.272873 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:49:17.272873 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:49:17.272873 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:49:17.272873 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:49:17.272873 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:49:17.272873 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:49:17.272873 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:49:17.272873 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:49:17.272873 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 19:49:17.272873 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 19:49:17.272873 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 19:49:17.272873 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 19:49:17.273744 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 19:49:17.273744 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 19:49:17.273744 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 19:49:17.273744 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 19:49:17.273744 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:49:17.273744 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:49:17.273744 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:49:17.273744 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:49:17.273744 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:49:17.273744 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:49:17.273744 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:49:17.273744 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Aug 05 19:49:17.273744 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:49:17.273744 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:49:17.273744 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:49:17.273744 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:49:17.274514 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:49:17.274514 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:49:17.274514 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:49:17.274514 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:49:17.274514 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 19:49:17.274514 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 19:49:17.274514 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 19:49:17.274514 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 19:49:17.274514 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 19:49:17.274514 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 19:49:17.274514 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 05 19:49:17.274514 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Aug 05 19:49:17.277377 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 172/341] 10.222.0.27 () {56 vars in 1114 bytes} [Fri Aug 5 19:49:17 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:49:17.285641 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce334104-adf3-4c81-9371-b0a662663515 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:17.289631 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7566cc3-2c9a-4651-a84b-9864bc993d52 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:17.349890 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9c33df-c308-44ef-ae5e-2c4dcabd1b78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:17.350532 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9c33df-c308-44ef-ae5e-2c4dcabd1b78 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:17.350820 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9c33df-c308-44ef-ae5e-2c4dcabd1b78 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:17.351100 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9c33df-c308-44ef-ae5e-2c4dcabd1b78 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:17.386632 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:17.451650 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf9c33df-c308-44ef-ae5e-2c4dcabd1b78 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:17.453759 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf9c33df-c308-44ef-ae5e-2c4dcabd1b78 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:17.500756 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7566cc3-2c9a-4651-a84b-9864bc993d52 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:17.525273 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:17.574618 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12602557-1e47-4f37-93bc-ce39d0640aaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:17.575248 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12602557-1e47-4f37-93bc-ce39d0640aaf None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:17.575524 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12602557-1e47-4f37-93bc-ce39d0640aaf None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:17.575794 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12602557-1e47-4f37-93bc-ce39d0640aaf None admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:17.576541 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12602557-1e47-4f37-93bc-ce39d0640aaf None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:49:17.576845 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12602557-1e47-4f37-93bc-ce39d0640aaf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:17.578305 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12602557-1e47-4f37-93bc-ce39d0640aaf None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:17.595407 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 173/342] 10.222.0.27 () {56 vars in 1111 bytes} [Fri Aug 5 19:49:17 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 306 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 19:49:17.596765 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef171cb0-fd0e-43f9-8d43-0f94722278d0 None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:17.597054 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef171cb0-fd0e-43f9-8d43-0f94722278d0 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:17.597329 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef171cb0-fd0e-43f9-8d43-0f94722278d0 None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:17.598767 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 174/343] 10.222.0.27 () {54 vars in 927 bytes} [Fri Aug 5 19:49:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:49:17.603475 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf831cae-4ea2-4e5e-9e9c-b237fea020c2 None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:17.603763 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf831cae-4ea2-4e5e-9e9c-b237fea020c2 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:17.604045 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf831cae-4ea2-4e5e-9e9c-b237fea020c2 None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:17.605575 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 175/344] 10.222.0.27 () {54 vars in 927 bytes} [Fri Aug 5 19:49:17 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:49:17.606533 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb74b71-3acb-4d09-95ea-2132580b993b None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:17.606820 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb74b71-3acb-4d09-95ea-2132580b993b None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:17.607097 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb74b71-3acb-4d09-95ea-2132580b993b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:17.607776 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 170/345] 10.222.0.27 () {58 vars in 1282 bytes} [Fri Aug 5 19:49:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 492 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 05 19:49:17.615719 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9c33df-c308-44ef-ae5e-2c4dcabd1b78 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:17.630633 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-cdb74b71-3acb-4d09-95ea-2132580b993b None None] MFA Rules not processed for user `03d1834a64cd4a43829827cb99a72747`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:49:17.631843 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-cdb74b71-3acb-4d09-95ea-2132580b993b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:49:17.715282 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 176/346] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Aug 5 19:49:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 19:49:17.717543 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc24e54d-b91f-4564-a927-8f84b3a8a1d2 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:17.717834 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc24e54d-b91f-4564-a927-8f84b3a8a1d2 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:17.718114 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc24e54d-b91f-4564-a927-8f84b3a8a1d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:17.745595 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-dc24e54d-b91f-4564-a927-8f84b3a8a1d2 None None] MFA Rules not processed for user `03d1834a64cd4a43829827cb99a72747`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:49:17.748079 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-dc24e54d-b91f-4564-a927-8f84b3a8a1d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:49:17.762961 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9c33df-c308-44ef-ae5e-2c4dcabd1b78 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:17.807967 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912a87ac-cc54-46c0-b165-3d6d4b702e1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:17.808650 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912a87ac-cc54-46c0-b165-3d6d4b702e1a None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:17.808936 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912a87ac-cc54-46c0-b165-3d6d4b702e1a None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:17.809200 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912a87ac-cc54-46c0-b165-3d6d4b702e1a None admin] PATH_INFO: `/v3/projects/6f67d76798fb469c8ec9dd2d185572c3` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:17.810455 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912a87ac-cc54-46c0-b165-3d6d4b702e1a None admin] RBAC: Authorizing `identity:get_project(project_id=6f67d76798fb469c8ec9dd2d185572c3)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:17.811923 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912a87ac-cc54-46c0-b165-3d6d4b702e1a None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:17.814046 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 171/347] 10.222.0.27 () {56 vars in 1186 bytes} [Fri Aug 5 19:49:17 2022] GET /identity/v3/projects/6f67d76798fb469c8ec9dd2d185572c3 => generated 298 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:49:17.814708 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912a87ac-cc54-46c0-b165-3d6d4b702e1a None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:17.833473 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:17.861562 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:17.882932 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 177/348] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Aug 5 19:49:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 19:49:17.884640 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc24e54d-b91f-4564-a927-8f84b3a8a1d2 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:17.960786 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912a87ac-cc54-46c0-b165-3d6d4b702e1a None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:18.007360 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0a24f5-7269-4176-9132-5fe966dfe4aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:18.007981 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0a24f5-7269-4176-9132-5fe966dfe4aa None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:18.008258 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0a24f5-7269-4176-9132-5fe966dfe4aa None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:18.008552 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0a24f5-7269-4176-9132-5fe966dfe4aa None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:18.018169 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc24e54d-b91f-4564-a927-8f84b3a8a1d2 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:18.057843 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b929586-cf9d-4b5c-9c93-520e03712438 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:18.058549 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b929586-cf9d-4b5c-9c93-520e03712438 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:18.058773 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b929586-cf9d-4b5c-9c93-520e03712438 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:18.059048 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b929586-cf9d-4b5c-9c93-520e03712438 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:18.071654 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b929586-cf9d-4b5c-9c93-520e03712438 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 19:49:18.076792 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-6b929586-cf9d-4b5c-9c93-520e03712438 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 05 19:49:18.076792 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:49:18.076792 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:49:18.076792 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:49:18.076792 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:49:18.076792 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:49:18.076792 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:49:18.076792 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:49:18.076792 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:49:18.076792 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:49:18.076792 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:49:18.076792 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:49:18.076792 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 19:49:18.076792 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 19:49:18.076792 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 19:49:18.076792 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 19:49:18.078971 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 19:49:18.078971 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 19:49:18.078971 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 19:49:18.078971 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 19:49:18.078971 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:49:18.078971 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:49:18.078971 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:49:18.078971 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:49:18.078971 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:49:18.078971 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:49:18.078971 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:49:18.078971 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Aug 05 19:49:18.078971 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:49:18.078971 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:49:18.078971 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:49:18.078971 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:49:18.080934 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:49:18.080934 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:49:18.080934 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:49:18.080934 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:49:18.080934 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 19:49:18.080934 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 19:49:18.080934 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 19:49:18.080934 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 19:49:18.080934 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 19:49:18.080934 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 19:49:18.080934 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 05 19:49:18.080934 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Aug 05 19:49:18.082390 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e0a24f5-7269-4176-9132-5fe966dfe4aa None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:18.082532 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 178/349] 10.222.0.27 () {56 vars in 1099 bytes} [Fri Aug 5 19:49:17 2022] GET /identity/v3/roles/reader => generated 84 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:49:18.086201 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b929586-cf9d-4b5c-9c93-520e03712438 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:18.086415 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e0a24f5-7269-4176-9132-5fe966dfe4aa None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:18.123515 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:18.132398 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:18.204379 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 172/350] 10.222.0.27 () {58 vars in 1282 bytes} [Fri Aug 5 19:49:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 390 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 05 19:49:18.205263 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0a24f5-7269-4176-9132-5fe966dfe4aa None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:18.227144 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b929586-cf9d-4b5c-9c93-520e03712438 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:18.292702 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c344c4-eb28-4c6a-b9ef-9a55f49241b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:18.293783 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c344c4-eb28-4c6a-b9ef-9a55f49241b7 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:18.294280 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c344c4-eb28-4c6a-b9ef-9a55f49241b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:18.294860 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c344c4-eb28-4c6a-b9ef-9a55f49241b7 None admin] PATH_INFO: `/v3/groups/da0669fc7bb94a6991321541b3f511e9` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:18.302998 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c344c4-eb28-4c6a-b9ef-9a55f49241b7 None admin] RBAC: Authorizing `identity:get_group(group_id=da0669fc7bb94a6991321541b3f511e9)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:18.304613 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c344c4-eb28-4c6a-b9ef-9a55f49241b7 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:18.306685 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 179/351] 10.222.0.27 () {56 vars in 1180 bytes} [Fri Aug 5 19:49:18 2022] GET /identity/v3/groups/da0669fc7bb94a6991321541b3f511e9 => generated 226 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:49:18.307542 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c344c4-eb28-4c6a-b9ef-9a55f49241b7 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:18.311348 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0a24f5-7269-4176-9132-5fe966dfe4aa None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:18.349663 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9494ab-d582-4cac-aff8-f11677962cd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:18.350306 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9494ab-d582-4cac-aff8-f11677962cd1 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:18.350594 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9494ab-d582-4cac-aff8-f11677962cd1 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:18.350854 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9494ab-d582-4cac-aff8-f11677962cd1 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:18.354762 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:18.368555 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db9494ab-d582-4cac-aff8-f11677962cd1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 19:49:18.373160 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application [None req-db9494ab-d582-4cac-aff8-f11677962cd1 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 05 19:49:18.373160 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:49:18.373160 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:49:18.373160 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:49:18.373160 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:49:18.373160 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:49:18.373160 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:49:18.373160 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:49:18.373160 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:49:18.373160 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:49:18.373160 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:49:18.373160 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:49:18.373160 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 19:49:18.373160 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 19:49:18.373160 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 19:49:18.373160 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 19:49:18.374043 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 19:49:18.374043 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 19:49:18.374043 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 19:49:18.374043 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 19:49:18.374043 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:49:18.374043 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:49:18.374043 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:49:18.374043 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:49:18.374043 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:49:18.374043 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:49:18.374043 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:49:18.374043 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application with Lock( Aug 05 19:49:18.374043 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:49:18.374043 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:49:18.374043 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:49:18.374043 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:49:18.374897 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:49:18.374897 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:49:18.374897 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:49:18.374897 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:49:18.374897 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 19:49:18.374897 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 19:49:18.374897 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 19:49:18.374897 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 19:49:18.374897 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 19:49:18.374897 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 19:49:18.374897 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 05 19:49:18.374897 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Aug 05 19:49:18.377778 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 173/352] 10.222.0.27 () {56 vars in 1099 bytes} [Fri Aug 5 19:49:18 2022] GET /identity/v3/roles/member => generated 84 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:49:18.378959 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9494ab-d582-4cac-aff8-f11677962cd1 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:18.421368 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c344c4-eb28-4c6a-b9ef-9a55f49241b7 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:18.425159 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:18.470757 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa0f2e9-a774-4fc0-a7e2-9cfb863df1f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:18.471469 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa0f2e9-a774-4fc0-a7e2-9cfb863df1f7 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:18.471930 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa0f2e9-a774-4fc0-a7e2-9cfb863df1f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:18.472353 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa0f2e9-a774-4fc0-a7e2-9cfb863df1f7 None admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:18.473250 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa0f2e9-a774-4fc0-a7e2-9cfb863df1f7 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:49:18.473803 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa0f2e9-a774-4fc0-a7e2-9cfb863df1f7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:18.475462 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa0f2e9-a774-4fc0-a7e2-9cfb863df1f7 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:18.489260 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 180/353] 10.222.0.27 () {56 vars in 1101 bytes} [Fri Aug 5 19:49:18 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:49:18.491324 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa0f2e9-a774-4fc0-a7e2-9cfb863df1f7 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:18.519037 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9494ab-d582-4cac-aff8-f11677962cd1 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:18.571979 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf80a8fb-c5a0-44f1-afad-8fa8387a7613 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:18.573027 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf80a8fb-c5a0-44f1-afad-8fa8387a7613 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:18.573526 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf80a8fb-c5a0-44f1-afad-8fa8387a7613 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:18.574003 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf80a8fb-c5a0-44f1-afad-8fa8387a7613 None admin] PATH_INFO: `/v3/projects/6f67d76798fb469c8ec9dd2d185572c3` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:18.576305 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf80a8fb-c5a0-44f1-afad-8fa8387a7613 None admin] RBAC: Authorizing `identity:get_project(project_id=6f67d76798fb469c8ec9dd2d185572c3)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:18.578556 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf80a8fb-c5a0-44f1-afad-8fa8387a7613 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:18.580797 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 174/354] 10.222.0.27 () {56 vars in 1186 bytes} [Fri Aug 5 19:49:18 2022] GET /identity/v3/projects/6f67d76798fb469c8ec9dd2d185572c3 => generated 298 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:49:18.581933 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf80a8fb-c5a0-44f1-afad-8fa8387a7613 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:18.627394 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa0f2e9-a774-4fc0-a7e2-9cfb863df1f7 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:18.664402 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:18.673224 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ccaffde-4afe-4737-a4f7-1fc102f1d89e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:18.673859 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccaffde-4afe-4737-a4f7-1fc102f1d89e None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:18.674143 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccaffde-4afe-4737-a4f7-1fc102f1d89e None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:18.675142 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccaffde-4afe-4737-a4f7-1fc102f1d89e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:18.675270 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ccaffde-4afe-4737-a4f7-1fc102f1d89e None admin] RBAC: Adding query filter params (group.id=da0669fc7bb94a6991321541b3f511e9, role.id=1a4f694d53054a24b580138b1ba77f3c, scope.project.id=6f67d76798fb469c8ec9dd2d185572c3) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:49:18.675520 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ccaffde-4afe-4737-a4f7-1fc102f1d89e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:18.680320 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ccaffde-4afe-4737-a4f7-1fc102f1d89e None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:18.685778 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 181/355] 10.222.0.27 () {56 vars in 1376 bytes} [Fri Aug 5 19:49:18 2022] GET /identity/v3/role_assignments?group.id=da0669fc7bb94a6991321541b3f511e9&scope.project.id=6f67d76798fb469c8ec9dd2d185572c3&role.id=1a4f694d53054a24b580138b1ba77f3c => generated 260 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:49:18.686553 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9385bd4c-d249-40d3-ac18-147fdd2c2ec5 None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:18.686819 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9385bd4c-d249-40d3-ac18-147fdd2c2ec5 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:18.687681 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9385bd4c-d249-40d3-ac18-147fdd2c2ec5 None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:18.689402 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 182/356] 10.222.0.27 () {54 vars in 927 bytes} [Fri Aug 5 19:49:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:49:18.690377 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9385bd4c-d249-40d3-ac18-147fdd2c2ec5 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:18.723831 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:18.725881 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf80a8fb-c5a0-44f1-afad-8fa8387a7613 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:18.777639 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057ea40b-cf44-4dfb-b4de-59e6c416a7f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:18.778370 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057ea40b-cf44-4dfb-b4de-59e6c416a7f8 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:18.778877 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057ea40b-cf44-4dfb-b4de-59e6c416a7f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:18.779318 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057ea40b-cf44-4dfb-b4de-59e6c416a7f8 None admin] PATH_INFO: `/v3/roles` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:18.780263 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057ea40b-cf44-4dfb-b4de-59e6c416a7f8 None admin] RBAC: Adding query filter params (name=member) {{(pid=80364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:49:18.781381 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057ea40b-cf44-4dfb-b4de-59e6c416a7f8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:18.784604 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057ea40b-cf44-4dfb-b4de-59e6c416a7f8 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:18.797698 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 175/357] 10.222.0.27 () {56 vars in 1101 bytes} [Fri Aug 5 19:49:18 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:49:18.799163 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057ea40b-cf44-4dfb-b4de-59e6c416a7f8 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:18.825133 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9385bd4c-d249-40d3-ac18-147fdd2c2ec5 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:18.864029 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d49d37f-af5a-4d0f-ac17-273d1793ec48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:18.864991 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d49d37f-af5a-4d0f-ac17-273d1793ec48 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:18.865417 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d49d37f-af5a-4d0f-ac17-273d1793ec48 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:18.865857 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d49d37f-af5a-4d0f-ac17-273d1793ec48 None admin] PATH_INFO: `/v3/users/bd35b64e464a49ce9a54e6a4723941c7` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:18.868172 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d49d37f-af5a-4d0f-ac17-273d1793ec48 None admin] RBAC: Authorizing `identity:get_user(user_id=bd35b64e464a49ce9a54e6a4723941c7)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:18.869972 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d49d37f-af5a-4d0f-ac17-273d1793ec48 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:18.871834 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 183/358] 10.222.0.27 () {56 vars in 1177 bytes} [Fri Aug 5 19:49:18 2022] GET /identity/v3/users/bd35b64e464a49ce9a54e6a4723941c7 => generated 293 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:49:18.873276 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af57639-2492-4979-85a1-28c6694547e9 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:18.873748 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af57639-2492-4979-85a1-28c6694547e9 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:18.874171 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af57639-2492-4979-85a1-28c6694547e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:18.896431 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-6af57639-2492-4979-85a1-28c6694547e9 None None] MFA Rules not processed for user `03d1834a64cd4a43829827cb99a72747`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:49:18.897900 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-6af57639-2492-4979-85a1-28c6694547e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:49:18.940170 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057ea40b-cf44-4dfb-b4de-59e6c416a7f8 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:18.969804 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:18.980090 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 184/359] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Aug 5 19:49:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 19:49:18.981231 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af57639-2492-4979-85a1-28c6694547e9 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:18.994278 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ac8967-5d63-46fc-b896-8833ad0f9a7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:18.995289 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ac8967-5d63-46fc-b896-8833ad0f9a7b None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:18.995741 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ac8967-5d63-46fc-b896-8833ad0f9a7b None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:18.996155 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ac8967-5d63-46fc-b896-8833ad0f9a7b None admin] PATH_INFO: `/v3/groups/da0669fc7bb94a6991321541b3f511e9` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:18.998361 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ac8967-5d63-46fc-b896-8833ad0f9a7b None admin] RBAC: Authorizing `identity:get_group(group_id=da0669fc7bb94a6991321541b3f511e9)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:19.000269 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ac8967-5d63-46fc-b896-8833ad0f9a7b None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:19.002252 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 176/360] 10.222.0.27 () {56 vars in 1180 bytes} [Fri Aug 5 19:49:18 2022] GET /identity/v3/groups/da0669fc7bb94a6991321541b3f511e9 => generated 226 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:49:19.003256 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ac8967-5d63-46fc-b896-8833ad0f9a7b None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:19.028811 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:19.128238 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af57639-2492-4979-85a1-28c6694547e9 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:19.147171 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ac8967-5d63-46fc-b896-8833ad0f9a7b None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:19.177058 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154eec13-074b-4b7b-879e-81700ca66ff8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:19.177561 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154eec13-074b-4b7b-879e-81700ca66ff8 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:19.177836 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154eec13-074b-4b7b-879e-81700ca66ff8 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:19.179562 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154eec13-074b-4b7b-879e-81700ca66ff8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:19.195003 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7127640c-10fa-4bf7-b877-08ac6a8b84d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:19.195777 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7127640c-10fa-4bf7-b877-08ac6a8b84d9 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:19.196064 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7127640c-10fa-4bf7-b877-08ac6a8b84d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:19.196566 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7127640c-10fa-4bf7-b877-08ac6a8b84d9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:19.198951 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7127640c-10fa-4bf7-b877-08ac6a8b84d9 None admin] RBAC: Adding query filter params (role.id=b885e5d9af42483799785b59b26b6207, user.id=bd35b64e464a49ce9a54e6a4723941c7, scope.system=all) {{(pid=80364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:49:19.199277 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7127640c-10fa-4bf7-b877-08ac6a8b84d9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:19.200844 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7127640c-10fa-4bf7-b877-08ac6a8b84d9 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:19.215383 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 177/361] 10.222.0.27 () {56 vars in 1308 bytes} [Fri Aug 5 19:49:19 2022] GET /identity/v3/role_assignments?user.id=bd35b64e464a49ce9a54e6a4723941c7&scope.system=all&role.id=b885e5d9af42483799785b59b26b6207 => generated 510 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:49:19.216177 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7127640c-10fa-4bf7-b877-08ac6a8b84d9 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:19.274580 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-154eec13-074b-4b7b-879e-81700ca66ff8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:19.276660 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-154eec13-074b-4b7b-879e-81700ca66ff8 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:19.284523 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:19.316289 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:19.353911 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7127640c-10fa-4bf7-b877-08ac6a8b84d9 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:19.394523 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7715ebd-511f-42b8-b3ab-9a7d923c591f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:19.395165 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7715ebd-511f-42b8-b3ab-9a7d923c591f None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:19.395507 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7715ebd-511f-42b8-b3ab-9a7d923c591f None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:19.395817 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7715ebd-511f-42b8-b3ab-9a7d923c591f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:19.396799 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7715ebd-511f-42b8-b3ab-9a7d923c591f None admin] RBAC: Adding query filter params (group.id=da0669fc7bb94a6991321541b3f511e9, role.id=dde0aa8028794132913bcba4d91858b8, scope.project.id=6f67d76798fb469c8ec9dd2d185572c3) {{(pid=80364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:49:19.399201 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7715ebd-511f-42b8-b3ab-9a7d923c591f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:19.399201 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7715ebd-511f-42b8-b3ab-9a7d923c591f None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:19.405556 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 178/362] 10.222.0.27 () {56 vars in 1376 bytes} [Fri Aug 5 19:49:19 2022] GET /identity/v3/role_assignments?group.id=da0669fc7bb94a6991321541b3f511e9&scope.project.id=6f67d76798fb469c8ec9dd2d185572c3&role.id=dde0aa8028794132913bcba4d91858b8 => generated 260 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:49:19.408022 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7715ebd-511f-42b8-b3ab-9a7d923c591f None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:19.416057 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 185/363] 10.222.0.27 () {58 vars in 1282 bytes} [Fri Aug 5 19:49:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 435 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 05 19:49:19.417852 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9b2773-feaf-4d9f-b8cf-debdc12831a2 None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:19.418371 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9b2773-feaf-4d9f-b8cf-debdc12831a2 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:19.423521 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9b2773-feaf-4d9f-b8cf-debdc12831a2 None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:19.425887 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 186/364] 10.222.0.27 () {54 vars in 927 bytes} [Fri Aug 5 19:49:19 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:49:19.426969 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9b2773-feaf-4d9f-b8cf-debdc12831a2 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:19.538134 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7715ebd-511f-42b8-b3ab-9a7d923c591f None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:19.556102 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:19.558613 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9b2773-feaf-4d9f-b8cf-debdc12831a2 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:19.590819 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167e6337-1a8b-479a-a426-ffa55655655e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:19.591485 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167e6337-1a8b-479a-a426-ffa55655655e None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:19.591876 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167e6337-1a8b-479a-a426-ffa55655655e None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:19.592458 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167e6337-1a8b-479a-a426-ffa55655655e None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:19.598631 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:19.612674 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-167e6337-1a8b-479a-a426-ffa55655655e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 19:49:19.614731 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ddb9bc3-f860-4d3b-9f3c-dc235f745baa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:19.616081 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddb9bc3-f860-4d3b-9f3c-dc235f745baa None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:19.616612 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddb9bc3-f860-4d3b-9f3c-dc235f745baa None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:19.617237 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddb9bc3-f860-4d3b-9f3c-dc235f745baa None admin] PATH_INFO: `/v3/projects/6440bf75a9134de7ba2c5fb9c23955e2` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:19.619243 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application [None req-167e6337-1a8b-479a-a426-ffa55655655e None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 05 19:49:19.619243 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:49:19.619243 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:49:19.619243 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:49:19.619243 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:49:19.619243 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:49:19.619243 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:49:19.619243 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:49:19.619243 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:49:19.619243 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:49:19.619243 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:49:19.619243 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:49:19.619243 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 19:49:19.619243 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 19:49:19.619243 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 19:49:19.619243 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 19:49:19.620298 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 19:49:19.620298 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 19:49:19.620298 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 19:49:19.620298 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 19:49:19.620298 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:49:19.620298 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:49:19.620298 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:49:19.620298 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:49:19.620298 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:49:19.620298 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:49:19.620298 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:49:19.620298 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application with Lock( Aug 05 19:49:19.620298 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:49:19.620298 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:49:19.620298 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:49:19.620298 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:49:19.621224 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:49:19.621224 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:49:19.621224 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:49:19.621224 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:49:19.621224 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 19:49:19.621224 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 19:49:19.621224 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 19:49:19.621224 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 19:49:19.621224 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 19:49:19.621224 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 19:49:19.621224 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 05 19:49:19.621224 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Aug 05 19:49:19.621847 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ddb9bc3-f860-4d3b-9f3c-dc235f745baa None admin] RBAC: Authorizing `identity:get_project(project_id=6440bf75a9134de7ba2c5fb9c23955e2)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:19.623233 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 179/365] 10.222.0.27 () {56 vars in 1114 bytes} [Fri Aug 5 19:49:19 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:49:19.625397 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da23a909-7d13-4cff-af23-b501dfa8f091 None None] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:19.625819 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ddb9bc3-f860-4d3b-9f3c-dc235f745baa None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:19.626074 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da23a909-7d13-4cff-af23-b501dfa8f091 None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:19.626487 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da23a909-7d13-4cff-af23-b501dfa8f091 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:19.627665 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 187/366] 10.222.0.27 () {56 vars in 1186 bytes} [Fri Aug 5 19:49:19 2022] GET /identity/v3/projects/6440bf75a9134de7ba2c5fb9c23955e2 => generated 302 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:49:19.633537 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ddb9bc3-f860-4d3b-9f3c-dc235f745baa None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:19.657684 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.auth.core [None req-da23a909-7d13-4cff-af23-b501dfa8f091 None None] MFA Rules not processed for user `03d1834a64cd4a43829827cb99a72747`. Rule list: `[]` (Enabled: `True`). {{(pid=80364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:49:19.658841 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-da23a909-7d13-4cff-af23-b501dfa8f091 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:49:19.756635 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 180/367] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Aug 5 19:49:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 19:49:19.757519 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da23a909-7d13-4cff-af23-b501dfa8f091 None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:19.763581 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ddb9bc3-f860-4d3b-9f3c-dc235f745baa None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:19.812602 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d09ee0-22de-4727-84d0-252067d0fcbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:19.813241 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d09ee0-22de-4727-84d0-252067d0fcbf None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:19.813566 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d09ee0-22de-4727-84d0-252067d0fcbf None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:19.813835 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d09ee0-22de-4727-84d0-252067d0fcbf None admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:19.814605 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d09ee0-22de-4727-84d0-252067d0fcbf None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:49:19.814994 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d09ee0-22de-4727-84d0-252067d0fcbf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:19.817853 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d09ee0-22de-4727-84d0-252067d0fcbf None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:19.828670 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 188/368] 10.222.0.27 () {56 vars in 1111 bytes} [Fri Aug 5 19:49:19 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:49:19.829335 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d09ee0-22de-4727-84d0-252067d0fcbf None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:19.879678 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da23a909-7d13-4cff-af23-b501dfa8f091 None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:19.882353 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:19.896308 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:19.922813 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d454ad54-6f40-4006-b9b2-e606427f7549 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:19.924154 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d454ad54-6f40-4006-b9b2-e606427f7549 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:19.924862 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d454ad54-6f40-4006-b9b2-e606427f7549 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:19.925531 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d454ad54-6f40-4006-b9b2-e606427f7549 None admin] PATH_INFO: `/v3/groups/da0669fc7bb94a6991321541b3f511e9` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:19.927819 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d454ad54-6f40-4006-b9b2-e606427f7549 None admin] RBAC: Authorizing `identity:get_group(group_id=da0669fc7bb94a6991321541b3f511e9)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:19.929911 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d454ad54-6f40-4006-b9b2-e606427f7549 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:19.932162 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 181/369] 10.222.0.27 () {56 vars in 1180 bytes} [Fri Aug 5 19:49:19 2022] GET /identity/v3/groups/da0669fc7bb94a6991321541b3f511e9 => generated 226 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:49:19.933155 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d454ad54-6f40-4006-b9b2-e606427f7549 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:19.949906 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d09ee0-22de-4727-84d0-252067d0fcbf None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:19.988780 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7307574-90c6-4a05-ad58-dd3c13a1585f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:19.989600 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7307574-90c6-4a05-ad58-dd3c13a1585f None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:19.989975 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7307574-90c6-4a05-ad58-dd3c13a1585f None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:19.990346 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7307574-90c6-4a05-ad58-dd3c13a1585f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:20.002131 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7307574-90c6-4a05-ad58-dd3c13a1585f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 19:49:20.008477 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-c7307574-90c6-4a05-ad58-dd3c13a1585f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 19:49:20.008477 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:49:20.008477 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:49:20.008477 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:49:20.008477 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:49:20.008477 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:49:20.008477 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:49:20.008477 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:49:20.008477 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:49:20.008477 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:49:20.008477 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:49:20.008477 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:49:20.008477 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 19:49:20.008477 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 19:49:20.008477 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 19:49:20.008477 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 19:49:20.009454 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 19:49:20.009454 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 19:49:20.009454 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 19:49:20.009454 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 19:49:20.009454 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:49:20.009454 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:49:20.009454 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:49:20.009454 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:49:20.009454 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:49:20.009454 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:49:20.009454 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:49:20.009454 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Aug 05 19:49:20.009454 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:49:20.009454 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:49:20.009454 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:49:20.009454 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:49:20.010236 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:49:20.010236 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:49:20.010236 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:49:20.010236 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:49:20.010236 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 19:49:20.010236 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 19:49:20.010236 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 19:49:20.010236 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 19:49:20.010236 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 19:49:20.010236 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 19:49:20.010236 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 19:49:20.010236 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Aug 05 19:49:20.010236 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 189/370] 10.222.0.27 () {56 vars in 1096 bytes} [Fri Aug 5 19:49:19 2022] GET /identity/v3/roles/admin => generated 83 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:49:20.011468 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7307574-90c6-4a05-ad58-dd3c13a1585f None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:20.045198 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d454ad54-6f40-4006-b9b2-e606427f7549 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:20.083333 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161da568-1cd1-458a-8ed2-574dce30e031 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:20.084104 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161da568-1cd1-458a-8ed2-574dce30e031 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:20.084529 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161da568-1cd1-458a-8ed2-574dce30e031 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:20.084875 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161da568-1cd1-458a-8ed2-574dce30e031 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:20.131783 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:20.145968 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7307574-90c6-4a05-ad58-dd3c13a1585f None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:20.165925 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-161da568-1cd1-458a-8ed2-574dce30e031 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:20.167766 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-161da568-1cd1-458a-8ed2-574dce30e031 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:20.168652 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:20.192581 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef63327-bad5-40eb-a05d-d43ca4cfd3b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:20.193238 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef63327-bad5-40eb-a05d-d43ca4cfd3b7 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:20.193518 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef63327-bad5-40eb-a05d-d43ca4cfd3b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:20.193788 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef63327-bad5-40eb-a05d-d43ca4cfd3b7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:20.195064 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef63327-bad5-40eb-a05d-d43ca4cfd3b7 None admin] RBAC: Adding query filter params (group.id=da0669fc7bb94a6991321541b3f511e9, role.id=1a4f694d53054a24b580138b1ba77f3c, scope.project.id=6440bf75a9134de7ba2c5fb9c23955e2) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:49:20.196080 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef63327-bad5-40eb-a05d-d43ca4cfd3b7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:20.198036 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef63327-bad5-40eb-a05d-d43ca4cfd3b7 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:20.205463 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 190/371] 10.222.0.27 () {56 vars in 1376 bytes} [Fri Aug 5 19:49:20 2022] GET /identity/v3/role_assignments?group.id=da0669fc7bb94a6991321541b3f511e9&scope.project.id=6440bf75a9134de7ba2c5fb9c23955e2&role.id=1a4f694d53054a24b580138b1ba77f3c => generated 260 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:49:20.206697 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef63327-bad5-40eb-a05d-d43ca4cfd3b7 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:20.312911 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 182/372] 10.222.0.27 () {58 vars in 1282 bytes} [Fri Aug 5 19:49:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 05 19:49:20.318476 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161da568-1cd1-458a-8ed2-574dce30e031 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:20.346893 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef63327-bad5-40eb-a05d-d43ca4cfd3b7 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:20.387308 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9b7bf7-f82a-42c5-a858-7ddbd2cd5054 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:20.388353 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9b7bf7-f82a-42c5-a858-7ddbd2cd5054 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:20.388856 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9b7bf7-f82a-42c5-a858-7ddbd2cd5054 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:20.389334 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9b7bf7-f82a-42c5-a858-7ddbd2cd5054 None admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:20.390293 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9b7bf7-f82a-42c5-a858-7ddbd2cd5054 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:49:20.390805 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9b7bf7-f82a-42c5-a858-7ddbd2cd5054 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:20.392363 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9b7bf7-f82a-42c5-a858-7ddbd2cd5054 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:20.401328 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:20.404305 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 191/373] 10.222.0.27 () {56 vars in 1099 bytes} [Fri Aug 5 19:49:20 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:49:20.410650 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9b7bf7-f82a-42c5-a858-7ddbd2cd5054 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:20.439055 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161da568-1cd1-458a-8ed2-574dce30e031 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:20.484992 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:20.485272 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d98e5b-135a-4b6f-b1be-c7c74e522dbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:20.486084 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d98e5b-135a-4b6f-b1be-c7c74e522dbd None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:20.486491 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d98e5b-135a-4b6f-b1be-c7c74e522dbd None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:20.486932 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d98e5b-135a-4b6f-b1be-c7c74e522dbd None admin] PATH_INFO: `/v3/projects/6440bf75a9134de7ba2c5fb9c23955e2` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:20.489503 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d98e5b-135a-4b6f-b1be-c7c74e522dbd None admin] RBAC: Authorizing `identity:get_project(project_id=6440bf75a9134de7ba2c5fb9c23955e2)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:20.491188 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d98e5b-135a-4b6f-b1be-c7c74e522dbd None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:20.493499 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 183/374] 10.222.0.27 () {56 vars in 1186 bytes} [Fri Aug 5 19:49:20 2022] GET /identity/v3/projects/6440bf75a9134de7ba2c5fb9c23955e2 => generated 302 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 19:49:20.499170 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d98e5b-135a-4b6f-b1be-c7c74e522dbd None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:20.619301 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9b7bf7-f82a-42c5-a858-7ddbd2cd5054 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:20.626797 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d98e5b-135a-4b6f-b1be-c7c74e522dbd None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:20.677945 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b5032a-2595-48a1-8be1-ffa49dedc80c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:20.678618 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b5032a-2595-48a1-8be1-ffa49dedc80c None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:20.678891 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b5032a-2595-48a1-8be1-ffa49dedc80c None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:20.679156 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b5032a-2595-48a1-8be1-ffa49dedc80c None admin] PATH_INFO: `/v3/groups/da0669fc7bb94a6991321541b3f511e9` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:20.683672 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ae21e4-87c7-48db-a2c2-f4da8539020c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:20.685296 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b5032a-2595-48a1-8be1-ffa49dedc80c None admin] RBAC: Authorizing `identity:get_group(group_id=da0669fc7bb94a6991321541b3f511e9)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:20.685792 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ae21e4-87c7-48db-a2c2-f4da8539020c None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:20.686603 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ae21e4-87c7-48db-a2c2-f4da8539020c None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:20.686816 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b5032a-2595-48a1-8be1-ffa49dedc80c None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:20.687685 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ae21e4-87c7-48db-a2c2-f4da8539020c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:20.689003 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 184/375] 10.222.0.27 () {56 vars in 1180 bytes} [Fri Aug 5 19:49:20 2022] GET /identity/v3/groups/da0669fc7bb94a6991321541b3f511e9 => generated 226 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:49:20.710752 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b5032a-2595-48a1-8be1-ffa49dedc80c None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:20.739219 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:20.823225 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ae21e4-87c7-48db-a2c2-f4da8539020c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:20.825119 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ae21e4-87c7-48db-a2c2-f4da8539020c None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:20.901785 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:20.964333 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b5032a-2595-48a1-8be1-ffa49dedc80c None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:21.024712 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 192/376] 10.222.0.27 () {58 vars in 1282 bytes} [Fri Aug 5 19:49:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 612 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 05 19:49:21.029788 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ae21e4-87c7-48db-a2c2-f4da8539020c None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:21.043038 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2967a759-90df-4aae-a5eb-3647707317ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:21.043667 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2967a759-90df-4aae-a5eb-3647707317ef None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:21.043944 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2967a759-90df-4aae-a5eb-3647707317ef None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:21.044215 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2967a759-90df-4aae-a5eb-3647707317ef None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:21.045522 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2967a759-90df-4aae-a5eb-3647707317ef None admin] RBAC: Adding query filter params (group.id=da0669fc7bb94a6991321541b3f511e9, role.id=dde0aa8028794132913bcba4d91858b8, scope.project.id=6440bf75a9134de7ba2c5fb9c23955e2) {{(pid=80364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:49:21.045829 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2967a759-90df-4aae-a5eb-3647707317ef None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:21.047836 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2967a759-90df-4aae-a5eb-3647707317ef None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:21.065556 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 185/377] 10.222.0.27 () {56 vars in 1376 bytes} [Fri Aug 5 19:49:20 2022] GET /identity/v3/role_assignments?group.id=da0669fc7bb94a6991321541b3f511e9&scope.project.id=6440bf75a9134de7ba2c5fb9c23955e2&role.id=dde0aa8028794132913bcba4d91858b8 => generated 260 bytes in 355 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 19:49:21.203121 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ae21e4-87c7-48db-a2c2-f4da8539020c None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:21.293879 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ad922d-a7e2-4fab-94d2-888029cdeb98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:21.295401 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ad922d-a7e2-4fab-94d2-888029cdeb98 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:21.296112 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ad922d-a7e2-4fab-94d2-888029cdeb98 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:21.297404 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ad922d-a7e2-4fab-94d2-888029cdeb98 None admin] PATH_INFO: `/v3/projects/d25cd7afe64843529ae31496a857a541` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:21.299983 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ad922d-a7e2-4fab-94d2-888029cdeb98 None admin] RBAC: Authorizing `identity:get_project(project_id=d25cd7afe64843529ae31496a857a541)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:21.303057 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ad922d-a7e2-4fab-94d2-888029cdeb98 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:21.307025 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 193/378] 10.222.0.27 () {56 vars in 1186 bytes} [Fri Aug 5 19:49:21 2022] GET /identity/v3/projects/d25cd7afe64843529ae31496a857a541 => generated 344 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:49:21.313040 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2967a759-90df-4aae-a5eb-3647707317ef None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:21.408463 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f440ebe-311b-41aa-8bee-f9c663733bcc None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:21.409070 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f440ebe-311b-41aa-8bee-f9c663733bcc None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:21.409549 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f440ebe-311b-41aa-8bee-f9c663733bcc None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:21.413193 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 194/379] 10.222.0.27 () {54 vars in 927 bytes} [Fri Aug 5 19:49:21 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:49:21.422187 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17753307-6284-49cc-bd95-4b6a3ab0501b None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:21.422642 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17753307-6284-49cc-bd95-4b6a3ab0501b None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:21.423096 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17753307-6284-49cc-bd95-4b6a3ab0501b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:21.447323 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:21.461595 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-17753307-6284-49cc-bd95-4b6a3ab0501b None None] MFA Rules not processed for user `03d1834a64cd4a43829827cb99a72747`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:49:21.463183 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-17753307-6284-49cc-bd95-4b6a3ab0501b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:49:21.471692 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:21.499067 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2967a759-90df-4aae-a5eb-3647707317ef None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:21.572680 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c58ffe3-dba5-481c-8b84-f6dfdde52f13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:21.573527 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c58ffe3-dba5-481c-8b84-f6dfdde52f13 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:21.573808 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c58ffe3-dba5-481c-8b84-f6dfdde52f13 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:21.574085 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c58ffe3-dba5-481c-8b84-f6dfdde52f13 None admin] PATH_INFO: `/v3/groups/291297a2f54e403d911b8e326c3201be` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:21.583309 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 195/380] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Aug 5 19:49:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 05 19:49:21.588389 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c58ffe3-dba5-481c-8b84-f6dfdde52f13 None admin] RBAC: Authorizing `identity:get_group(group_id=291297a2f54e403d911b8e326c3201be)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:21.590170 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c58ffe3-dba5-481c-8b84-f6dfdde52f13 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:21.596637 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 186/381] 10.222.0.27 () {56 vars in 1180 bytes} [Fri Aug 5 19:49:21 2022] GET /identity/v3/groups/291297a2f54e403d911b8e326c3201be => generated 229 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:49:21.599296 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17753307-6284-49cc-bd95-4b6a3ab0501b None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:21.616824 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c58ffe3-dba5-481c-8b84-f6dfdde52f13 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:21.805004 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17753307-6284-49cc-bd95-4b6a3ab0501b None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:21.811004 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c58ffe3-dba5-481c-8b84-f6dfdde52f13 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:21.877069 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751355ec-1169-4f60-a7c4-bed21331b478 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:21.878688 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751355ec-1169-4f60-a7c4-bed21331b478 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:21.879371 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751355ec-1169-4f60-a7c4-bed21331b478 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:21.880413 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751355ec-1169-4f60-a7c4-bed21331b478 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:21.887260 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5d90bf-6002-4512-b05e-6bd21ca0c2f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:21.887836 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:21.889537 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5d90bf-6002-4512-b05e-6bd21ca0c2f3 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:21.892387 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5d90bf-6002-4512-b05e-6bd21ca0c2f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:21.892960 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5d90bf-6002-4512-b05e-6bd21ca0c2f3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:21.894084 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc5d90bf-6002-4512-b05e-6bd21ca0c2f3 None admin] RBAC: Adding query filter params (group.id=291297a2f54e403d911b8e326c3201be, role.id=c173688a960b46e5be65f0ccd6fbd734, scope.project.id=d25cd7afe64843529ae31496a857a541) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:49:21.894840 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc5d90bf-6002-4512-b05e-6bd21ca0c2f3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:21.897259 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc5d90bf-6002-4512-b05e-6bd21ca0c2f3 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:21.907408 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-751355ec-1169-4f60-a7c4-bed21331b478 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 19:49:21.916661 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 196/382] 10.222.0.27 () {56 vars in 1376 bytes} [Fri Aug 5 19:49:21 2022] GET /identity/v3/role_assignments?group.id=291297a2f54e403d911b8e326c3201be&scope.project.id=d25cd7afe64843529ae31496a857a541&role.id=c173688a960b46e5be65f0ccd6fbd734 => generated 260 bytes in 314 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:49:21.921272 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application [None req-751355ec-1169-4f60-a7c4-bed21331b478 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 05 19:49:21.921272 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:49:21.921272 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:49:21.921272 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:49:21.921272 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:49:21.921272 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:49:21.921272 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:49:21.921272 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:49:21.921272 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:49:21.921272 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:49:21.921272 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:49:21.921272 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:49:21.921272 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 19:49:21.921272 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 19:49:21.921272 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 19:49:21.921272 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 19:49:21.922216 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 19:49:21.922216 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 19:49:21.922216 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 19:49:21.922216 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 19:49:21.922216 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:49:21.922216 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:49:21.922216 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:49:21.922216 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:49:21.922216 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:49:21.922216 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:49:21.922216 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:49:21.922216 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application with Lock( Aug 05 19:49:21.922216 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:49:21.922216 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:49:21.922216 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:49:21.922216 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:49:21.923083 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:49:21.923083 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:49:21.923083 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:49:21.923083 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:49:21.923083 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 19:49:21.923083 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 19:49:21.923083 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 19:49:21.923083 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 19:49:21.923083 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 19:49:21.923083 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 19:49:21.923083 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 05 19:49:21.923083 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Aug 05 19:49:21.926720 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 187/383] 10.222.0.27 () {56 vars in 1099 bytes} [Fri Aug 5 19:49:21 2022] GET /identity/v3/roles/member => generated 84 bytes in 309 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:49:21.934254 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5d90bf-6002-4512-b05e-6bd21ca0c2f3 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:21.939257 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:22.058941 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5d90bf-6002-4512-b05e-6bd21ca0c2f3 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:22.097661 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf7e7cf-6d33-41e6-a185-56d2bc868f84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:22.098766 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf7e7cf-6d33-41e6-a185-56d2bc868f84 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:22.099256 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf7e7cf-6d33-41e6-a185-56d2bc868f84 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:22.099785 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf7e7cf-6d33-41e6-a185-56d2bc868f84 None admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:22.100863 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf7e7cf-6d33-41e6-a185-56d2bc868f84 None admin] RBAC: Adding query filter params (name=member) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:49:22.101425 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf7e7cf-6d33-41e6-a185-56d2bc868f84 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:22.103205 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf7e7cf-6d33-41e6-a185-56d2bc868f84 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:22.113738 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 197/384] 10.222.0.27 () {56 vars in 1101 bytes} [Fri Aug 5 19:49:21 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:49:22.121758 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751355ec-1169-4f60-a7c4-bed21331b478 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:22.253329 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751355ec-1169-4f60-a7c4-bed21331b478 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:22.295818 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1027646-21c6-4362-96de-c1d83927296f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:22.297589 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1027646-21c6-4362-96de-c1d83927296f None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:22.298114 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1027646-21c6-4362-96de-c1d83927296f None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:22.298606 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1027646-21c6-4362-96de-c1d83927296f None admin] PATH_INFO: `/v3/groups/da0669fc7bb94a6991321541b3f511e9` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:22.300063 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1027646-21c6-4362-96de-c1d83927296f None admin] RBAC: Authorizing `identity:get_group(group_id=da0669fc7bb94a6991321541b3f511e9)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:22.301957 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1027646-21c6-4362-96de-c1d83927296f None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:22.303976 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 188/385] 10.222.0.27 () {56 vars in 1180 bytes} [Fri Aug 5 19:49:22 2022] GET /identity/v3/groups/da0669fc7bb94a6991321541b3f511e9 => generated 226 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:49:22.309441 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf7e7cf-6d33-41e6-a185-56d2bc868f84 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:22.384889 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:22.421257 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf7e7cf-6d33-41e6-a185-56d2bc868f84 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:22.458773 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f06186-3fe2-4e79-9d29-2424cded1bb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:22.459480 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f06186-3fe2-4e79-9d29-2424cded1bb2 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:22.459765 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f06186-3fe2-4e79-9d29-2424cded1bb2 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:22.460089 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f06186-3fe2-4e79-9d29-2424cded1bb2 None admin] PATH_INFO: `/v3/projects/6f67d76798fb469c8ec9dd2d185572c3` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:22.461313 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f06186-3fe2-4e79-9d29-2424cded1bb2 None admin] RBAC: Authorizing `identity:get_project(project_id=6f67d76798fb469c8ec9dd2d185572c3)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:22.462823 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f06186-3fe2-4e79-9d29-2424cded1bb2 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:22.464930 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 198/386] 10.222.0.27 () {56 vars in 1186 bytes} [Fri Aug 5 19:49:22 2022] GET /identity/v3/projects/6f67d76798fb469c8ec9dd2d185572c3 => generated 298 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 19:49:22.470974 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1027646-21c6-4362-96de-c1d83927296f None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:22.496921 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed1fe2e-3298-4fc5-9613-da21519e9148 None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:22.497255 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed1fe2e-3298-4fc5-9613-da21519e9148 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:22.497475 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed1fe2e-3298-4fc5-9613-da21519e9148 None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:22.499194 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 199/387] 10.222.0.27 () {54 vars in 927 bytes} [Fri Aug 5 19:49:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 05 19:49:22.506481 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e45311f-e1a3-4fa0-842a-2aeb7de0b025 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:22.506889 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e45311f-e1a3-4fa0-842a-2aeb7de0b025 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:22.507372 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e45311f-e1a3-4fa0-842a-2aeb7de0b025 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:22.531258 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-3e45311f-e1a3-4fa0-842a-2aeb7de0b025 None None] MFA Rules not processed for user `03d1834a64cd4a43829827cb99a72747`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:49:22.532251 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-3e45311f-e1a3-4fa0-842a-2aeb7de0b025 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:49:22.551587 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:22.599493 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1027646-21c6-4362-96de-c1d83927296f None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:22.614755 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 200/388] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Aug 5 19:49:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 05 19:49:22.640526 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e45311f-e1a3-4fa0-842a-2aeb7de0b025 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:22.643349 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81fa872-5661-465c-83ed-d5c844ad5859 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:22.644239 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81fa872-5661-465c-83ed-d5c844ad5859 None admin] REQUEST_METHOD: `PUT` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:22.644721 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81fa872-5661-465c-83ed-d5c844ad5859 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:22.645350 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81fa872-5661-465c-83ed-d5c844ad5859 None admin] PATH_INFO: `/v3/projects/6f67d76798fb469c8ec9dd2d185572c3/groups/da0669fc7bb94a6991321541b3f511e9/roles/1a4f694d53054a24b580138b1ba77f3c` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:22.653978 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a81fa872-5661-465c-83ed-d5c844ad5859 None admin] RBAC: Authorizing `identity:create_grant(project_id=6f67d76798fb469c8ec9dd2d185572c3, group_id=da0669fc7bb94a6991321541b3f511e9, role_id=1a4f694d53054a24b580138b1ba77f3c)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:22.656248 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a81fa872-5661-465c-83ed-d5c844ad5859 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:22.678877 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 189/389] 10.222.0.27 () {58 vars in 1442 bytes} [Fri Aug 5 19:49:22 2022] PUT /identity/v3/projects/6f67d76798fb469c8ec9dd2d185572c3/groups/da0669fc7bb94a6991321541b3f511e9/roles/1a4f694d53054a24b580138b1ba77f3c => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 19:49:22.713053 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:22.762861 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e45311f-e1a3-4fa0-842a-2aeb7de0b025 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:22.776348 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fb09c8-53ad-49b2-82e8-2e5e7a286610 None None] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:22.776685 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fb09c8-53ad-49b2-82e8-2e5e7a286610 None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:22.776931 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fb09c8-53ad-49b2-82e8-2e5e7a286610 None None] PATH_INFO: `/` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:22.778604 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 190/390] 10.222.0.27 () {54 vars in 927 bytes} [Fri Aug 5 19:49:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:49:22.783985 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4216b4f9-687c-41e2-9b89-ed432d6964c5 None None] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:22.784315 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4216b4f9-687c-41e2-9b89-ed432d6964c5 None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:22.784553 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4216b4f9-687c-41e2-9b89-ed432d6964c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:22.805491 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324a3596-fcea-4d18-a6fc-6af1f21fe833 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:22.806170 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324a3596-fcea-4d18-a6fc-6af1f21fe833 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:22.806419 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324a3596-fcea-4d18-a6fc-6af1f21fe833 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:22.806701 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324a3596-fcea-4d18-a6fc-6af1f21fe833 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:22.820866 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-324a3596-fcea-4d18-a6fc-6af1f21fe833 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 19:49:22.824807 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.auth.core [None req-4216b4f9-687c-41e2-9b89-ed432d6964c5 None None] MFA Rules not processed for user `03d1834a64cd4a43829827cb99a72747`. Rule list: `[]` (Enabled: `True`). {{(pid=80364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:49:22.826028 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-4216b4f9-687c-41e2-9b89-ed432d6964c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:49:22.826335 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-324a3596-fcea-4d18-a6fc-6af1f21fe833 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 05 19:49:22.826335 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:49:22.826335 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:49:22.826335 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:49:22.826335 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:49:22.826335 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:49:22.826335 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:49:22.826335 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:49:22.826335 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:49:22.826335 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:49:22.826335 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:49:22.826335 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:49:22.826335 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 19:49:22.826335 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 19:49:22.826335 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 19:49:22.826335 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 19:49:22.827172 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 19:49:22.827172 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 19:49:22.827172 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 19:49:22.827172 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 19:49:22.827172 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:49:22.827172 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:49:22.827172 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:49:22.827172 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:49:22.827172 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:49:22.827172 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:49:22.827172 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:49:22.827172 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Aug 05 19:49:22.827172 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:49:22.827172 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:49:22.827172 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:49:22.827172 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:49:22.828576 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:49:22.828576 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:49:22.828576 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:49:22.828576 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:49:22.828576 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 19:49:22.828576 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 19:49:22.828576 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 19:49:22.828576 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 19:49:22.828576 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 19:49:22.828576 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 19:49:22.828576 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 05 19:49:22.828576 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Aug 05 19:49:22.832007 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 201/391] 10.222.0.27 () {56 vars in 1114 bytes} [Fri Aug 5 19:49:22 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 05 19:49:22.835104 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324a3596-fcea-4d18-a6fc-6af1f21fe833 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:22.960955 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324a3596-fcea-4d18-a6fc-6af1f21fe833 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:23.004965 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 191/392] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Aug 5 19:49:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 19:49:23.009333 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:23.018485 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc7fbb8-d0dc-4bad-9eb1-56b6d19445f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:23.019054 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc7fbb8-d0dc-4bad-9eb1-56b6d19445f4 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:23.019342 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc7fbb8-d0dc-4bad-9eb1-56b6d19445f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:23.019609 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc7fbb8-d0dc-4bad-9eb1-56b6d19445f4 None admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:23.020336 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc7fbb8-d0dc-4bad-9eb1-56b6d19445f4 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:49:23.020673 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc7fbb8-d0dc-4bad-9eb1-56b6d19445f4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:23.022060 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc7fbb8-d0dc-4bad-9eb1-56b6d19445f4 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:23.032597 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 202/393] 10.222.0.27 () {56 vars in 1111 bytes} [Fri Aug 5 19:49:22 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:49:23.037240 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4216b4f9-687c-41e2-9b89-ed432d6964c5 None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:23.038693 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc7fbb8-d0dc-4bad-9eb1-56b6d19445f4 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:23.139271 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:23.189794 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc7fbb8-d0dc-4bad-9eb1-56b6d19445f4 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:23.230000 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64110db1-ec63-4b59-bdbf-e503ee2e5ac5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:23.231350 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64110db1-ec63-4b59-bdbf-e503ee2e5ac5 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:23.231667 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64110db1-ec63-4b59-bdbf-e503ee2e5ac5 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:23.232123 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64110db1-ec63-4b59-bdbf-e503ee2e5ac5 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:23.234837 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4216b4f9-687c-41e2-9b89-ed432d6964c5 None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:23.245141 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64110db1-ec63-4b59-bdbf-e503ee2e5ac5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 19:49:23.251522 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-64110db1-ec63-4b59-bdbf-e503ee2e5ac5 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 05 19:49:23.251522 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:49:23.251522 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:49:23.251522 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:49:23.251522 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:49:23.251522 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:49:23.251522 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:49:23.251522 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:49:23.251522 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:49:23.251522 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:49:23.251522 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:49:23.251522 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:49:23.251522 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 19:49:23.251522 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 19:49:23.251522 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 19:49:23.251522 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 19:49:23.252414 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 19:49:23.252414 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 19:49:23.252414 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 19:49:23.252414 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 19:49:23.252414 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:49:23.252414 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:49:23.252414 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:49:23.252414 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:49:23.252414 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:49:23.252414 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:49:23.252414 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:49:23.252414 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Aug 05 19:49:23.252414 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:49:23.252414 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:49:23.252414 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:49:23.252414 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:49:23.253224 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:49:23.253224 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:49:23.253224 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:49:23.253224 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:49:23.253224 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 19:49:23.253224 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 19:49:23.253224 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 19:49:23.253224 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 19:49:23.253224 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 19:49:23.253224 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 19:49:23.253224 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 05 19:49:23.253224 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Aug 05 19:49:23.253224 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 203/394] 10.222.0.27 () {56 vars in 1099 bytes} [Fri Aug 5 19:49:23 2022] GET /identity/v3/roles/member => generated 84 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 05 19:49:23.259861 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64110db1-ec63-4b59-bdbf-e503ee2e5ac5 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:23.294537 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8d3c8f-502b-4d4f-a635-f5139c73f2ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:23.297499 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8d3c8f-502b-4d4f-a635-f5139c73f2ed None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:23.298800 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8d3c8f-502b-4d4f-a635-f5139c73f2ed None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:23.302342 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8d3c8f-502b-4d4f-a635-f5139c73f2ed None admin] PATH_INFO: `/v3/groups/da0669fc7bb94a6991321541b3f511e9` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:23.304970 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed8d3c8f-502b-4d4f-a635-f5139c73f2ed None admin] RBAC: Authorizing `identity:get_group(group_id=da0669fc7bb94a6991321541b3f511e9)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:23.306649 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed8d3c8f-502b-4d4f-a635-f5139c73f2ed None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:23.308571 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 192/395] 10.222.0.27 () {56 vars in 1180 bytes} [Fri Aug 5 19:49:23 2022] GET /identity/v3/groups/da0669fc7bb94a6991321541b3f511e9 => generated 226 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:49:23.317577 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8d3c8f-502b-4d4f-a635-f5139c73f2ed None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:23.379304 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:23.465550 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8d3c8f-502b-4d4f-a635-f5139c73f2ed None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:23.501058 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:23.507719 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9e1c49-390b-4947-8d52-c7d73aed95e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:23.508738 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9e1c49-390b-4947-8d52-c7d73aed95e4 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:23.509376 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9e1c49-390b-4947-8d52-c7d73aed95e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:23.509875 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9e1c49-390b-4947-8d52-c7d73aed95e4 None admin] PATH_INFO: `/v3/projects/6f67d76798fb469c8ec9dd2d185572c3` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:23.511273 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9e1c49-390b-4947-8d52-c7d73aed95e4 None admin] RBAC: Authorizing `identity:get_project(project_id=6f67d76798fb469c8ec9dd2d185572c3)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:23.512961 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9e1c49-390b-4947-8d52-c7d73aed95e4 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:23.520854 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 193/396] 10.222.0.27 () {56 vars in 1186 bytes} [Fri Aug 5 19:49:23 2022] GET /identity/v3/projects/6f67d76798fb469c8ec9dd2d185572c3 => generated 298 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:49:23.522390 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9e1c49-390b-4947-8d52-c7d73aed95e4 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:23.534782 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64110db1-ec63-4b59-bdbf-e503ee2e5ac5 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:23.630588 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76aa1203-fc31-4820-a1a7-e95add0fafc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:23.631825 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76aa1203-fc31-4820-a1a7-e95add0fafc3 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:23.632072 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76aa1203-fc31-4820-a1a7-e95add0fafc3 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:23.632594 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76aa1203-fc31-4820-a1a7-e95add0fafc3 None admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:23.633391 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76aa1203-fc31-4820-a1a7-e95add0fafc3 None admin] RBAC: Adding query filter params (name=member) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:49:23.633854 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76aa1203-fc31-4820-a1a7-e95add0fafc3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:23.635161 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76aa1203-fc31-4820-a1a7-e95add0fafc3 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:23.646287 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9e1c49-390b-4947-8d52-c7d73aed95e4 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:23.652642 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 204/397] 10.222.0.27 () {56 vars in 1101 bytes} [Fri Aug 5 19:49:23 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 393 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:49:23.657886 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76aa1203-fc31-4820-a1a7-e95add0fafc3 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:23.690117 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c019de-c69b-46db-9206-20fe936aee74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:23.692198 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c019de-c69b-46db-9206-20fe936aee74 None admin] REQUEST_METHOD: `PUT` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:23.693605 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c019de-c69b-46db-9206-20fe936aee74 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:23.694274 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c019de-c69b-46db-9206-20fe936aee74 None admin] PATH_INFO: `/v3/projects/6f67d76798fb469c8ec9dd2d185572c3/groups/da0669fc7bb94a6991321541b3f511e9/roles/dde0aa8028794132913bcba4d91858b8` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:23.702086 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c019de-c69b-46db-9206-20fe936aee74 None admin] RBAC: Authorizing `identity:create_grant(project_id=6f67d76798fb469c8ec9dd2d185572c3, group_id=da0669fc7bb94a6991321541b3f511e9, role_id=dde0aa8028794132913bcba4d91858b8)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:23.704421 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c019de-c69b-46db-9206-20fe936aee74 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:23.735120 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 194/398] 10.222.0.27 () {58 vars in 1442 bytes} [Fri Aug 5 19:49:23 2022] PUT /identity/v3/projects/6f67d76798fb469c8ec9dd2d185572c3/groups/da0669fc7bb94a6991321541b3f511e9/roles/dde0aa8028794132913bcba4d91858b8 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 19:49:23.822723 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76aa1203-fc31-4820-a1a7-e95add0fafc3 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:23.835861 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:23.865002 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475db127-a4a2-4e2d-b55f-e75f76f26b3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:23.865664 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475db127-a4a2-4e2d-b55f-e75f76f26b3f None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:23.865945 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475db127-a4a2-4e2d-b55f-e75f76f26b3f None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:23.866223 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475db127-a4a2-4e2d-b55f-e75f76f26b3f None admin] PATH_INFO: `/v3/groups/da0669fc7bb94a6991321541b3f511e9` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:23.867451 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-475db127-a4a2-4e2d-b55f-e75f76f26b3f None admin] RBAC: Authorizing `identity:get_group(group_id=da0669fc7bb94a6991321541b3f511e9)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:23.868964 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-475db127-a4a2-4e2d-b55f-e75f76f26b3f None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:23.871068 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 205/399] 10.222.0.27 () {56 vars in 1180 bytes} [Fri Aug 5 19:49:23 2022] GET /identity/v3/groups/da0669fc7bb94a6991321541b3f511e9 => generated 226 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 19:49:23.877267 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c019de-c69b-46db-9206-20fe936aee74 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:23.933376 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:23.988902 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c019de-c69b-46db-9206-20fe936aee74 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:24.026401 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364018f8-b543-461b-82bc-384f70ed8f3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:24.027050 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364018f8-b543-461b-82bc-384f70ed8f3e None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:24.027327 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364018f8-b543-461b-82bc-384f70ed8f3e None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:24.027603 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364018f8-b543-461b-82bc-384f70ed8f3e None admin] PATH_INFO: `/v3/projects/6440bf75a9134de7ba2c5fb9c23955e2` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:24.028833 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364018f8-b543-461b-82bc-384f70ed8f3e None admin] RBAC: Authorizing `identity:get_project(project_id=6440bf75a9134de7ba2c5fb9c23955e2)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:24.030290 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364018f8-b543-461b-82bc-384f70ed8f3e None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:24.036621 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 195/400] 10.222.0.27 () {56 vars in 1186 bytes} [Fri Aug 5 19:49:23 2022] GET /identity/v3/projects/6440bf75a9134de7ba2c5fb9c23955e2 => generated 302 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:49:24.039099 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475db127-a4a2-4e2d-b55f-e75f76f26b3f None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:24.202892 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475db127-a4a2-4e2d-b55f-e75f76f26b3f None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:24.263332 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3de841-c412-4db8-8761-f055fc5c3202 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:24.264411 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3de841-c412-4db8-8761-f055fc5c3202 None admin] REQUEST_METHOD: `PUT` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:24.264919 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3de841-c412-4db8-8761-f055fc5c3202 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:24.265468 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3de841-c412-4db8-8761-f055fc5c3202 None admin] PATH_INFO: `/v3/projects/6440bf75a9134de7ba2c5fb9c23955e2/groups/da0669fc7bb94a6991321541b3f511e9/roles/1a4f694d53054a24b580138b1ba77f3c` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:24.267884 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e3de841-c412-4db8-8761-f055fc5c3202 None admin] RBAC: Authorizing `identity:create_grant(project_id=6440bf75a9134de7ba2c5fb9c23955e2, group_id=da0669fc7bb94a6991321541b3f511e9, role_id=1a4f694d53054a24b580138b1ba77f3c)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:24.269644 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e3de841-c412-4db8-8761-f055fc5c3202 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:24.287150 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 206/401] 10.222.0.27 () {58 vars in 1442 bytes} [Fri Aug 5 19:49:24 2022] PUT /identity/v3/projects/6440bf75a9134de7ba2c5fb9c23955e2/groups/da0669fc7bb94a6991321541b3f511e9/roles/1a4f694d53054a24b580138b1ba77f3c => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 19:49:24.324940 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587678e6-0a7d-4183-aece-51980708dfd5 None None] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:24.325263 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587678e6-0a7d-4183-aece-51980708dfd5 None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:24.325492 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587678e6-0a7d-4183-aece-51980708dfd5 None None] PATH_INFO: `/` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:24.327140 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 196/402] 10.222.0.27 () {54 vars in 927 bytes} [Fri Aug 5 19:49:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:49:24.334096 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288b7afa-5a4a-40c1-8569-ac2ad373a1e2 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:24.334709 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288b7afa-5a4a-40c1-8569-ac2ad373a1e2 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:24.335190 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288b7afa-5a4a-40c1-8569-ac2ad373a1e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:24.361143 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-288b7afa-5a4a-40c1-8569-ac2ad373a1e2 None None] MFA Rules not processed for user `03d1834a64cd4a43829827cb99a72747`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:49:24.362781 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-288b7afa-5a4a-40c1-8569-ac2ad373a1e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:49:24.412629 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:24.471005 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 207/403] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Aug 5 19:49:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 19:49:24.508594 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587678e6-0a7d-4183-aece-51980708dfd5 None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:24.622139 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587678e6-0a7d-4183-aece-51980708dfd5 None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:24.649314 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:24.665279 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a17add-2187-4afb-8fef-94b1edaa6ca2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:24.665928 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a17add-2187-4afb-8fef-94b1edaa6ca2 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:24.666208 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a17add-2187-4afb-8fef-94b1edaa6ca2 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:24.666481 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a17add-2187-4afb-8fef-94b1edaa6ca2 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:24.677558 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a17add-2187-4afb-8fef-94b1edaa6ca2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 19:49:24.682528 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application [None req-58a17add-2187-4afb-8fef-94b1edaa6ca2 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 05 19:49:24.682528 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:49:24.682528 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:49:24.682528 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:49:24.682528 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:49:24.682528 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:49:24.682528 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:49:24.682528 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:49:24.682528 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:49:24.682528 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:49:24.682528 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:49:24.682528 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:49:24.682528 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 19:49:24.682528 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 19:49:24.682528 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 19:49:24.682528 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 19:49:24.683410 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 19:49:24.683410 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 19:49:24.683410 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 19:49:24.683410 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 19:49:24.683410 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:49:24.683410 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:49:24.683410 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:49:24.683410 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:49:24.683410 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:49:24.683410 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:49:24.683410 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:49:24.683410 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application with Lock( Aug 05 19:49:24.683410 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:49:24.683410 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:49:24.683410 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:49:24.683410 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:49:24.684209 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:49:24.684209 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:49:24.684209 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:49:24.684209 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:49:24.684209 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 19:49:24.684209 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 19:49:24.684209 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 19:49:24.684209 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 19:49:24.684209 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 19:49:24.684209 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 19:49:24.684209 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 05 19:49:24.684209 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Aug 05 19:49:24.688605 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 197/404] 10.222.0.27 () {56 vars in 1114 bytes} [Fri Aug 5 19:49:24 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:49:24.690503 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288b7afa-5a4a-40c1-8569-ac2ad373a1e2 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:24.835627 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288b7afa-5a4a-40c1-8569-ac2ad373a1e2 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:24.889772 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a6e3c3-ac9b-4408-96c8-2c7e817642f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:24.891330 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a6e3c3-ac9b-4408-96c8-2c7e817642f3 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:24.891823 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a6e3c3-ac9b-4408-96c8-2c7e817642f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:24.892316 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a6e3c3-ac9b-4408-96c8-2c7e817642f3 None admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:24.893322 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a6e3c3-ac9b-4408-96c8-2c7e817642f3 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:49:24.893848 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a6e3c3-ac9b-4408-96c8-2c7e817642f3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:24.895446 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a6e3c3-ac9b-4408-96c8-2c7e817642f3 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:24.908667 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 208/405] 10.222.0.27 () {56 vars in 1111 bytes} [Fri Aug 5 19:49:24 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:49:24.914048 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a17add-2187-4afb-8fef-94b1edaa6ca2 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:24.989943 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9cb7d1-193c-43c8-a006-75f4ebb7fb3a None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:24.990317 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9cb7d1-193c-43c8-a006-75f4ebb7fb3a None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:24.990586 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9cb7d1-193c-43c8-a006-75f4ebb7fb3a None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:24.992315 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 209/406] 10.222.0.27 () {54 vars in 927 bytes} [Fri Aug 5 19:49:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 05 19:49:24.998299 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037e9a50-5988-4a1a-8199-352bcc2c0f54 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:24.998620 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037e9a50-5988-4a1a-8199-352bcc2c0f54 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:24.998843 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037e9a50-5988-4a1a-8199-352bcc2c0f54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:25.018552 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-037e9a50-5988-4a1a-8199-352bcc2c0f54 None None] MFA Rules not processed for user `03d1834a64cd4a43829827cb99a72747`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:49:25.019748 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-037e9a50-5988-4a1a-8199-352bcc2c0f54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:49:25.042237 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a17add-2187-4afb-8fef-94b1edaa6ca2 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:25.053648 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:25.103726 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48473805-ee6d-4dd2-8e12-02b1519a2dad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:25.104356 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48473805-ee6d-4dd2-8e12-02b1519a2dad None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:25.104667 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48473805-ee6d-4dd2-8e12-02b1519a2dad None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:25.104941 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48473805-ee6d-4dd2-8e12-02b1519a2dad None admin] PATH_INFO: `/v3/groups/da0669fc7bb94a6991321541b3f511e9` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:25.106273 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48473805-ee6d-4dd2-8e12-02b1519a2dad None admin] RBAC: Authorizing `identity:get_group(group_id=da0669fc7bb94a6991321541b3f511e9)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:25.107780 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48473805-ee6d-4dd2-8e12-02b1519a2dad None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:25.114751 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 198/407] 10.222.0.27 () {56 vars in 1180 bytes} [Fri Aug 5 19:49:24 2022] GET /identity/v3/groups/da0669fc7bb94a6991321541b3f511e9 => generated 226 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:49:25.116008 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48473805-ee6d-4dd2-8e12-02b1519a2dad None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:25.130431 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 210/408] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Aug 5 19:49:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 19:49:25.157792 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037e9a50-5988-4a1a-8199-352bcc2c0f54 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:25.234637 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:25.313293 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037e9a50-5988-4a1a-8199-352bcc2c0f54 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:25.353239 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48473805-ee6d-4dd2-8e12-02b1519a2dad None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:25.353446 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714ccf85-935a-4c7c-947b-0a64fea25a09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:25.354089 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714ccf85-935a-4c7c-947b-0a64fea25a09 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:25.354328 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714ccf85-935a-4c7c-947b-0a64fea25a09 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:25.354573 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714ccf85-935a-4c7c-947b-0a64fea25a09 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:25.367754 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-714ccf85-935a-4c7c-947b-0a64fea25a09 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 19:49:25.376525 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-714ccf85-935a-4c7c-947b-0a64fea25a09 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 19:49:25.376525 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:49:25.376525 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:49:25.376525 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:49:25.376525 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:49:25.376525 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:49:25.376525 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:49:25.376525 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:49:25.376525 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:49:25.376525 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:49:25.376525 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:49:25.376525 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:49:25.376525 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 19:49:25.376525 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 19:49:25.376525 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 19:49:25.376525 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 19:49:25.377475 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 19:49:25.377475 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 19:49:25.377475 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 19:49:25.377475 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 19:49:25.377475 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:49:25.377475 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:49:25.377475 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:49:25.377475 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:49:25.377475 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:49:25.377475 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:49:25.377475 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:49:25.377475 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Aug 05 19:49:25.377475 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:49:25.377475 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:49:25.377475 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:49:25.377475 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:49:25.378275 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:49:25.378275 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:49:25.378275 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:49:25.378275 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:49:25.378275 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 19:49:25.378275 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 19:49:25.378275 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 19:49:25.378275 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 19:49:25.378275 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 19:49:25.378275 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 19:49:25.378275 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 19:49:25.378275 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Aug 05 19:49:25.378275 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 211/409] 10.222.0.27 () {56 vars in 1096 bytes} [Fri Aug 5 19:49:25 2022] GET /identity/v3/roles/admin => generated 83 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 05 19:49:25.380649 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714ccf85-935a-4c7c-947b-0a64fea25a09 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:25.415939 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:25.428868 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c495aaa-cf51-42cb-aac3-76b42df56cf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:25.429535 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c495aaa-cf51-42cb-aac3-76b42df56cf4 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:25.429813 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c495aaa-cf51-42cb-aac3-76b42df56cf4 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:25.430091 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c495aaa-cf51-42cb-aac3-76b42df56cf4 None admin] PATH_INFO: `/v3/projects/6440bf75a9134de7ba2c5fb9c23955e2` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:25.431357 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c495aaa-cf51-42cb-aac3-76b42df56cf4 None admin] RBAC: Authorizing `identity:get_project(project_id=6440bf75a9134de7ba2c5fb9c23955e2)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:25.433322 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c495aaa-cf51-42cb-aac3-76b42df56cf4 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:25.435608 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 199/410] 10.222.0.27 () {56 vars in 1186 bytes} [Fri Aug 5 19:49:25 2022] GET /identity/v3/projects/6440bf75a9134de7ba2c5fb9c23955e2 => generated 302 bytes in 320 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:49:25.437532 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37536aa-d3a5-4c37-a751-7c796c0cda11 None None] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:25.437816 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37536aa-d3a5-4c37-a751-7c796c0cda11 None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:25.438088 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37536aa-d3a5-4c37-a751-7c796c0cda11 None None] PATH_INFO: `/` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:25.440548 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 200/411] 10.222.0.27 () {54 vars in 927 bytes} [Fri Aug 5 19:49:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:49:25.441747 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37536aa-d3a5-4c37-a751-7c796c0cda11 None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:25.520307 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714ccf85-935a-4c7c-947b-0a64fea25a09 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:25.559925 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3542691-0f5b-4229-b47e-d8c2283a955e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:25.560578 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3542691-0f5b-4229-b47e-d8c2283a955e None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:25.560948 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3542691-0f5b-4229-b47e-d8c2283a955e None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:25.561221 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3542691-0f5b-4229-b47e-d8c2283a955e None admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:25.562479 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3542691-0f5b-4229-b47e-d8c2283a955e None admin] RBAC: Adding query filter params (name=admin) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:49:25.562781 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3542691-0f5b-4229-b47e-d8c2283a955e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:25.564232 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3542691-0f5b-4229-b47e-d8c2283a955e None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:25.565196 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37536aa-d3a5-4c37-a751-7c796c0cda11 None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:25.573473 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 212/412] 10.222.0.27 () {56 vars in 1099 bytes} [Fri Aug 5 19:49:25 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 19:49:25.574837 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcfc253-e3f6-42b2-8b1a-74b943a5b1df None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:25.575125 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcfc253-e3f6-42b2-8b1a-74b943a5b1df None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:25.575399 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcfc253-e3f6-42b2-8b1a-74b943a5b1df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:25.581790 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:25.597744 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-4bcfc253-e3f6-42b2-8b1a-74b943a5b1df None None] MFA Rules not processed for user `03d1834a64cd4a43829827cb99a72747`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:49:25.599079 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-4bcfc253-e3f6-42b2-8b1a-74b943a5b1df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:49:25.605350 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aec0237-c3f3-48df-99ef-38564b041d21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:25.606000 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aec0237-c3f3-48df-99ef-38564b041d21 None admin] REQUEST_METHOD: `PUT` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:25.606282 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aec0237-c3f3-48df-99ef-38564b041d21 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:25.606653 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aec0237-c3f3-48df-99ef-38564b041d21 None admin] PATH_INFO: `/v3/projects/6440bf75a9134de7ba2c5fb9c23955e2/groups/da0669fc7bb94a6991321541b3f511e9/roles/dde0aa8028794132913bcba4d91858b8` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:25.609306 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aec0237-c3f3-48df-99ef-38564b041d21 None admin] RBAC: Authorizing `identity:create_grant(project_id=6440bf75a9134de7ba2c5fb9c23955e2, group_id=da0669fc7bb94a6991321541b3f511e9, role_id=dde0aa8028794132913bcba4d91858b8)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:25.610925 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aec0237-c3f3-48df-99ef-38564b041d21 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:25.629138 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 201/413] 10.222.0.27 () {58 vars in 1442 bytes} [Fri Aug 5 19:49:25 2022] PUT /identity/v3/projects/6440bf75a9134de7ba2c5fb9c23955e2/groups/da0669fc7bb94a6991321541b3f511e9/roles/dde0aa8028794132913bcba4d91858b8 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 05 19:49:25.630643 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aec0237-c3f3-48df-99ef-38564b041d21 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:25.683040 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 213/414] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Aug 5 19:49:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 19:49:25.716414 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bcfc253-e3f6-42b2-8b1a-74b943a5b1df None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:25.729864 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:25.768347 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aec0237-c3f3-48df-99ef-38564b041d21 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:25.818166 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398194d7-7265-4a4b-82da-14fa9a25f9cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:25.818902 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398194d7-7265-4a4b-82da-14fa9a25f9cb None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:25.819182 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398194d7-7265-4a4b-82da-14fa9a25f9cb None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:25.819460 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398194d7-7265-4a4b-82da-14fa9a25f9cb None admin] PATH_INFO: `/v3/groups/291297a2f54e403d911b8e326c3201be` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:25.820689 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398194d7-7265-4a4b-82da-14fa9a25f9cb None admin] RBAC: Authorizing `identity:get_group(group_id=291297a2f54e403d911b8e326c3201be)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:25.822900 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398194d7-7265-4a4b-82da-14fa9a25f9cb None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:25.825014 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 202/415] 10.222.0.27 () {56 vars in 1180 bytes} [Fri Aug 5 19:49:25 2022] GET /identity/v3/groups/291297a2f54e403d911b8e326c3201be => generated 229 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:49:25.833041 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398194d7-7265-4a4b-82da-14fa9a25f9cb None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:25.891611 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bcfc253-e3f6-42b2-8b1a-74b943a5b1df None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:25.894573 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:25.944700 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6750bb-d859-4a30-9779-52309fe68a11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:25.945691 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6750bb-d859-4a30-9779-52309fe68a11 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:25.946113 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6750bb-d859-4a30-9779-52309fe68a11 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:25.946552 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6750bb-d859-4a30-9779-52309fe68a11 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:25.960430 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c6750bb-d859-4a30-9779-52309fe68a11 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 19:49:25.965769 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-8c6750bb-d859-4a30-9779-52309fe68a11 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 05 19:49:25.965769 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:49:25.965769 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:49:25.965769 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:49:25.965769 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:49:25.965769 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:49:25.965769 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:49:25.965769 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:49:25.965769 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:49:25.965769 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:49:25.965769 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:49:25.965769 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:49:25.965769 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 19:49:25.965769 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 19:49:25.965769 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 19:49:25.965769 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 19:49:25.966672 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 19:49:25.966672 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 19:49:25.966672 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 19:49:25.966672 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 19:49:25.966672 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:49:25.966672 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:49:25.966672 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:49:25.966672 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:49:25.966672 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:49:25.966672 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:49:25.966672 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:49:25.966672 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Aug 05 19:49:25.966672 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:49:25.966672 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:49:25.966672 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:49:25.966672 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:49:25.967453 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:49:25.967453 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:49:25.967453 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:49:25.967453 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:49:25.967453 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 19:49:25.967453 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 19:49:25.967453 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 19:49:25.967453 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 19:49:25.967453 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 19:49:25.967453 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 19:49:25.967453 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 05 19:49:25.967453 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Aug 05 19:49:25.969797 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 214/416] 10.222.0.27 () {56 vars in 1099 bytes} [Fri Aug 5 19:49:25 2022] GET /identity/v3/roles/member => generated 84 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 05 19:49:25.976186 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6750bb-d859-4a30-9779-52309fe68a11 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:25.977259 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398194d7-7265-4a4b-82da-14fa9a25f9cb None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:26.022169 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee2f280-2798-445e-884d-01a78ca00a45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:26.022867 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee2f280-2798-445e-884d-01a78ca00a45 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:26.023151 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee2f280-2798-445e-884d-01a78ca00a45 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:26.023426 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee2f280-2798-445e-884d-01a78ca00a45 None admin] PATH_INFO: `/v3/projects/d25cd7afe64843529ae31496a857a541` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:26.025562 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee2f280-2798-445e-884d-01a78ca00a45 None admin] RBAC: Authorizing `identity:get_project(project_id=d25cd7afe64843529ae31496a857a541)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:26.027008 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee2f280-2798-445e-884d-01a78ca00a45 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:26.030096 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 203/417] 10.222.0.27 () {56 vars in 1186 bytes} [Fri Aug 5 19:49:25 2022] GET /identity/v3/projects/d25cd7afe64843529ae31496a857a541 => generated 344 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 19:49:26.035424 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee2f280-2798-445e-884d-01a78ca00a45 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:26.074218 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:26.141991 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6750bb-d859-4a30-9779-52309fe68a11 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:26.199632 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee2f280-2798-445e-884d-01a78ca00a45 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:26.220837 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:26.224149 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7270072-22a6-46bb-bc97-5b8a38a3b339 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:26.225003 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7270072-22a6-46bb-bc97-5b8a38a3b339 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:26.225422 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7270072-22a6-46bb-bc97-5b8a38a3b339 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:26.225836 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7270072-22a6-46bb-bc97-5b8a38a3b339 None admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:26.226695 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7270072-22a6-46bb-bc97-5b8a38a3b339 None admin] RBAC: Adding query filter params (name=member) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:49:26.227139 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7270072-22a6-46bb-bc97-5b8a38a3b339 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:26.229010 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7270072-22a6-46bb-bc97-5b8a38a3b339 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:26.247988 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 215/418] 10.222.0.27 () {56 vars in 1101 bytes} [Fri Aug 5 19:49:25 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:49:26.255043 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7270072-22a6-46bb-bc97-5b8a38a3b339 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:26.273421 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae33e1a1-b152-42bc-8502-71efcbc70bec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:26.274329 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae33e1a1-b152-42bc-8502-71efcbc70bec None admin] REQUEST_METHOD: `PUT` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:26.275097 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae33e1a1-b152-42bc-8502-71efcbc70bec None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:26.276647 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae33e1a1-b152-42bc-8502-71efcbc70bec None admin] PATH_INFO: `/v3/projects/d25cd7afe64843529ae31496a857a541/groups/291297a2f54e403d911b8e326c3201be/roles/c173688a960b46e5be65f0ccd6fbd734` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:26.281721 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae33e1a1-b152-42bc-8502-71efcbc70bec None admin] RBAC: Authorizing `identity:create_grant(project_id=d25cd7afe64843529ae31496a857a541, group_id=291297a2f54e403d911b8e326c3201be, role_id=c173688a960b46e5be65f0ccd6fbd734)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:26.284404 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae33e1a1-b152-42bc-8502-71efcbc70bec None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:26.305757 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 204/419] 10.222.0.27 () {58 vars in 1442 bytes} [Fri Aug 5 19:49:26 2022] PUT /identity/v3/projects/d25cd7afe64843529ae31496a857a541/groups/291297a2f54e403d911b8e326c3201be/roles/c173688a960b46e5be65f0ccd6fbd734 => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 05 19:49:26.434254 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7270072-22a6-46bb-bc97-5b8a38a3b339 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:26.476469 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:26.483702 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba338e96-b680-4831-a3fb-84d24f9648ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:26.484470 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba338e96-b680-4831-a3fb-84d24f9648ee None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:26.484782 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba338e96-b680-4831-a3fb-84d24f9648ee None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:26.485057 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba338e96-b680-4831-a3fb-84d24f9648ee None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:26.557789 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba338e96-b680-4831-a3fb-84d24f9648ee None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:26.559260 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba338e96-b680-4831-a3fb-84d24f9648ee None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:26.565273 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27212ad4-9463-4ed9-8938-4fc5e1b0d8ec None None] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:26.565555 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27212ad4-9463-4ed9-8938-4fc5e1b0d8ec None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:26.565828 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27212ad4-9463-4ed9-8938-4fc5e1b0d8ec None None] PATH_INFO: `/` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:26.567307 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 205/420] 10.222.0.27 () {54 vars in 927 bytes} [Fri Aug 5 19:49:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 05 19:49:26.572864 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1c1b17-e99f-470e-99a0-8a7d1664e06d None None] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:26.573429 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1c1b17-e99f-470e-99a0-8a7d1664e06d None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:26.573902 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1c1b17-e99f-470e-99a0-8a7d1664e06d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:26.594601 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.auth.core [None req-1d1c1b17-e99f-470e-99a0-8a7d1664e06d None None] MFA Rules not processed for user `03d1834a64cd4a43829827cb99a72747`. Rule list: `[]` (Enabled: `True`). {{(pid=80364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:49:26.596129 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-1d1c1b17-e99f-470e-99a0-8a7d1664e06d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:49:26.662170 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 216/421] 10.222.0.27 () {58 vars in 1282 bytes} [Fri Aug 5 19:49:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 407 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 05 19:49:26.669511 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba338e96-b680-4831-a3fb-84d24f9648ee None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:26.689268 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 206/422] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Aug 5 19:49:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 05 19:49:26.716853 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:26.744030 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1c1b17-e99f-470e-99a0-8a7d1664e06d None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:26.821732 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba338e96-b680-4831-a3fb-84d24f9648ee None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:26.862782 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8716e3f-ce41-4402-b240-d5edb24c8b67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:26.863905 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8716e3f-ce41-4402-b240-d5edb24c8b67 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:26.865168 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8716e3f-ce41-4402-b240-d5edb24c8b67 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:26.866284 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8716e3f-ce41-4402-b240-d5edb24c8b67 None admin] PATH_INFO: `/v3/projects/6f67d76798fb469c8ec9dd2d185572c3` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:26.868859 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8716e3f-ce41-4402-b240-d5edb24c8b67 None admin] RBAC: Authorizing `identity:get_project(project_id=6f67d76798fb469c8ec9dd2d185572c3)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:26.870789 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8716e3f-ce41-4402-b240-d5edb24c8b67 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:26.880639 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 217/423] 10.222.0.27 () {56 vars in 1186 bytes} [Fri Aug 5 19:49:26 2022] GET /identity/v3/projects/6f67d76798fb469c8ec9dd2d185572c3 => generated 298 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:49:26.883584 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8716e3f-ce41-4402-b240-d5edb24c8b67 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:26.941420 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:26.947143 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1c1b17-e99f-470e-99a0-8a7d1664e06d None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:26.992398 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e7d160-22e8-44d0-b264-0486bf6eed99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:26.993171 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e7d160-22e8-44d0-b264-0486bf6eed99 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:26.993518 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e7d160-22e8-44d0-b264-0486bf6eed99 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:26.993805 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e7d160-22e8-44d0-b264-0486bf6eed99 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:27.008549 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e7d160-22e8-44d0-b264-0486bf6eed99 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 19:49:27.011122 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:27.014967 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8716e3f-ce41-4402-b240-d5edb24c8b67 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:27.015662 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application [None req-34e7d160-22e8-44d0-b264-0486bf6eed99 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 05 19:49:27.015662 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:49:27.015662 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:49:27.015662 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:49:27.015662 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:49:27.015662 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:49:27.015662 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:49:27.015662 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:49:27.015662 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:49:27.015662 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:49:27.015662 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:49:27.015662 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:49:27.015662 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 19:49:27.015662 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 19:49:27.015662 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 19:49:27.015662 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 19:49:27.016580 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 19:49:27.016580 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 19:49:27.016580 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 19:49:27.016580 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 19:49:27.016580 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:49:27.016580 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:49:27.016580 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:49:27.016580 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:49:27.016580 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:49:27.016580 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:49:27.016580 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:49:27.016580 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application with Lock( Aug 05 19:49:27.016580 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:49:27.016580 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:49:27.016580 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:49:27.016580 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:49:27.017447 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:49:27.017447 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:49:27.017447 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:49:27.017447 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:49:27.017447 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 19:49:27.017447 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 19:49:27.017447 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 19:49:27.017447 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 19:49:27.017447 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 19:49:27.017447 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 19:49:27.017447 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 05 19:49:27.017447 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Aug 05 19:49:27.017447 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 207/424] 10.222.0.27 () {56 vars in 1114 bytes} [Fri Aug 5 19:49:26 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 274 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:49:27.023194 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d0f8ab-e46f-4660-be90-561717a1206b None None] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:27.023194 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d0f8ab-e46f-4660-be90-561717a1206b None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:27.023194 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d0f8ab-e46f-4660-be90-561717a1206b None None] PATH_INFO: `/` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:27.023194 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 208/425] 10.222.0.27 () {54 vars in 927 bytes} [Fri Aug 5 19:49:27 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:49:27.025391 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d0f8ab-e46f-4660-be90-561717a1206b None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:27.063676 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d323c2a1-cc14-43a1-b61e-4877a67b5448 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:27.064755 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d323c2a1-cc14-43a1-b61e-4877a67b5448 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:27.065253 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d323c2a1-cc14-43a1-b61e-4877a67b5448 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:27.065751 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d323c2a1-cc14-43a1-b61e-4877a67b5448 None admin] PATH_INFO: `/v3/groups/da0669fc7bb94a6991321541b3f511e9` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:27.067318 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d323c2a1-cc14-43a1-b61e-4877a67b5448 None admin] RBAC: Authorizing `identity:get_group(group_id=da0669fc7bb94a6991321541b3f511e9)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:27.068988 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d323c2a1-cc14-43a1-b61e-4877a67b5448 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:27.076049 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 218/426] 10.222.0.27 () {56 vars in 1180 bytes} [Fri Aug 5 19:49:26 2022] GET /identity/v3/groups/da0669fc7bb94a6991321541b3f511e9 => generated 226 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:49:27.077856 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c13f83-6886-4aef-86a0-2af34d9689d5 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:27.078364 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c13f83-6886-4aef-86a0-2af34d9689d5 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:27.078849 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c13f83-6886-4aef-86a0-2af34d9689d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:27.098664 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-44c13f83-6886-4aef-86a0-2af34d9689d5 None None] MFA Rules not processed for user `03d1834a64cd4a43829827cb99a72747`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:49:27.099943 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-44c13f83-6886-4aef-86a0-2af34d9689d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:49:27.158252 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d0f8ab-e46f-4660-be90-561717a1206b None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:27.177558 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 219/427] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Aug 5 19:49:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 19:49:27.178501 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c13f83-6886-4aef-86a0-2af34d9689d5 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:27.222036 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98acd3e9-df57-421c-be31-3546e708a7ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:27.222600 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98acd3e9-df57-421c-be31-3546e708a7ab None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:27.223259 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98acd3e9-df57-421c-be31-3546e708a7ab None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:27.223536 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98acd3e9-df57-421c-be31-3546e708a7ab None admin] PATH_INFO: `/v3/roles` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:27.224534 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98acd3e9-df57-421c-be31-3546e708a7ab None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:49:27.224861 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98acd3e9-df57-421c-be31-3546e708a7ab None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:27.226925 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98acd3e9-df57-421c-be31-3546e708a7ab None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:27.237805 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 209/428] 10.222.0.27 () {56 vars in 1111 bytes} [Fri Aug 5 19:49:27 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:49:27.238710 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98acd3e9-df57-421c-be31-3546e708a7ab None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:27.271187 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:27.305645 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:27.310742 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c13f83-6886-4aef-86a0-2af34d9689d5 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:27.348510 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b295181-aeab-499d-b796-d62e9a0e5b41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:27.349317 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b295181-aeab-499d-b796-d62e9a0e5b41 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:27.349698 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b295181-aeab-499d-b796-d62e9a0e5b41 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:27.350083 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b295181-aeab-499d-b796-d62e9a0e5b41 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:27.351069 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b295181-aeab-499d-b796-d62e9a0e5b41 None admin] RBAC: Adding query filter params (group.id=da0669fc7bb94a6991321541b3f511e9, role.id=1a4f694d53054a24b580138b1ba77f3c, scope.project.id=6f67d76798fb469c8ec9dd2d185572c3) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:49:27.351545 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b295181-aeab-499d-b796-d62e9a0e5b41 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:27.353171 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b295181-aeab-499d-b796-d62e9a0e5b41 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:27.363101 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 220/429] 10.222.0.27 () {56 vars in 1376 bytes} [Fri Aug 5 19:49:27 2022] GET /identity/v3/role_assignments?group.id=da0669fc7bb94a6991321541b3f511e9&scope.project.id=6f67d76798fb469c8ec9dd2d185572c3&role.id=1a4f694d53054a24b580138b1ba77f3c => generated 611 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:49:27.364103 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b295181-aeab-499d-b796-d62e9a0e5b41 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:27.387696 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98acd3e9-df57-421c-be31-3546e708a7ab None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:27.430835 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47af8df1-8bd9-40f6-8102-ca507effe135 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:27.431497 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47af8df1-8bd9-40f6-8102-ca507effe135 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:27.431966 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47af8df1-8bd9-40f6-8102-ca507effe135 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:27.432244 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47af8df1-8bd9-40f6-8102-ca507effe135 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:27.456513 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47af8df1-8bd9-40f6-8102-ca507effe135 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 19:49:27.465559 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application [None req-47af8df1-8bd9-40f6-8102-ca507effe135 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 05 19:49:27.465559 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:49:27.465559 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:49:27.465559 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:49:27.465559 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:49:27.465559 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:49:27.465559 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:49:27.465559 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:49:27.465559 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:49:27.465559 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:49:27.465559 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:49:27.465559 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:49:27.465559 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 19:49:27.465559 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 19:49:27.465559 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 19:49:27.465559 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 19:49:27.466543 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 19:49:27.466543 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 19:49:27.466543 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 19:49:27.466543 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 19:49:27.466543 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:49:27.466543 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:49:27.466543 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:49:27.466543 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:49:27.466543 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:49:27.466543 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:49:27.466543 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:49:27.466543 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application with Lock( Aug 05 19:49:27.466543 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:49:27.466543 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:49:27.466543 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:49:27.466543 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:49:27.467328 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:49:27.467328 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:49:27.467328 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:49:27.467328 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:49:27.467328 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 19:49:27.467328 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 19:49:27.467328 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 19:49:27.467328 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 19:49:27.467328 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 19:49:27.467328 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 19:49:27.467328 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 05 19:49:27.467328 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Aug 05 19:49:27.471878 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 210/430] 10.222.0.27 () {56 vars in 1099 bytes} [Fri Aug 5 19:49:27 2022] GET /identity/v3/roles/member => generated 84 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:49:27.479421 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47af8df1-8bd9-40f6-8102-ca507effe135 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:27.586964 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b295181-aeab-499d-b796-d62e9a0e5b41 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:27.597500 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:27.623414 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47af8df1-8bd9-40f6-8102-ca507effe135 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:27.626840 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f904c0f0-2a02-46f8-bce0-6f7476105c42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:27.627546 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f904c0f0-2a02-46f8-bce0-6f7476105c42 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:27.627891 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f904c0f0-2a02-46f8-bce0-6f7476105c42 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:27.628227 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f904c0f0-2a02-46f8-bce0-6f7476105c42 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:27.668191 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:27.673740 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7965ae48-e1e2-4ccb-b528-cf3d0e3d7a93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:27.673740 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7965ae48-e1e2-4ccb-b528-cf3d0e3d7a93 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:27.676308 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7965ae48-e1e2-4ccb-b528-cf3d0e3d7a93 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:27.676308 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7965ae48-e1e2-4ccb-b528-cf3d0e3d7a93 None admin] PATH_INFO: `/v3/roles` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:27.676308 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7965ae48-e1e2-4ccb-b528-cf3d0e3d7a93 None admin] RBAC: Adding query filter params (name=member) {{(pid=80364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:49:27.676308 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7965ae48-e1e2-4ccb-b528-cf3d0e3d7a93 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:27.680703 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7965ae48-e1e2-4ccb-b528-cf3d0e3d7a93 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:27.690360 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 211/431] 10.222.0.27 () {56 vars in 1101 bytes} [Fri Aug 5 19:49:27 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 19:49:27.700418 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7965ae48-e1e2-4ccb-b528-cf3d0e3d7a93 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:27.730850 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f904c0f0-2a02-46f8-bce0-6f7476105c42 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:27.732349 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f904c0f0-2a02-46f8-bce0-6f7476105c42 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:27.836811 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7965ae48-e1e2-4ccb-b528-cf3d0e3d7a93 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:27.845885 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 221/432] 10.222.0.27 () {58 vars in 1282 bytes} [Fri Aug 5 19:49:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 482 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 05 19:49:27.851814 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f904c0f0-2a02-46f8-bce0-6f7476105c42 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:27.882943 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f113fc73-d6d4-49ce-b588-fd0bae8f9eba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:27.883825 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f113fc73-d6d4-49ce-b588-fd0bae8f9eba None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:27.884283 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f113fc73-d6d4-49ce-b588-fd0bae8f9eba None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:27.884819 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f113fc73-d6d4-49ce-b588-fd0bae8f9eba None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:27.904476 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:27.929784 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:27.963434 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f113fc73-d6d4-49ce-b588-fd0bae8f9eba None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:27.965266 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f113fc73-d6d4-49ce-b588-fd0bae8f9eba None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:27.974038 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f904c0f0-2a02-46f8-bce0-6f7476105c42 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:28.017663 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97b26ef-c523-4f8a-a759-e010d08eec79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:28.018273 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97b26ef-c523-4f8a-a759-e010d08eec79 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:28.018504 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97b26ef-c523-4f8a-a759-e010d08eec79 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:28.018778 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97b26ef-c523-4f8a-a759-e010d08eec79 None admin] PATH_INFO: `/v3/projects/6f67d76798fb469c8ec9dd2d185572c3` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:28.020350 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b97b26ef-c523-4f8a-a759-e010d08eec79 None admin] RBAC: Authorizing `identity:get_project(project_id=6f67d76798fb469c8ec9dd2d185572c3)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:28.022112 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b97b26ef-c523-4f8a-a759-e010d08eec79 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:28.024574 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 222/433] 10.222.0.27 () {56 vars in 1186 bytes} [Fri Aug 5 19:49:27 2022] GET /identity/v3/projects/6f67d76798fb469c8ec9dd2d185572c3 => generated 298 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:49:28.030312 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97b26ef-c523-4f8a-a759-e010d08eec79 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:28.105764 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 212/434] 10.222.0.27 () {58 vars in 1282 bytes} [Fri Aug 5 19:49:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 405 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 05 19:49:28.108368 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f113fc73-d6d4-49ce-b588-fd0bae8f9eba None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:28.171716 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97b26ef-c523-4f8a-a759-e010d08eec79 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:28.172353 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:28.215041 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:28.225490 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a0b228-31f9-43e8-84bf-18e37ac2ff53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:28.226905 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a0b228-31f9-43e8-84bf-18e37ac2ff53 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:28.227645 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a0b228-31f9-43e8-84bf-18e37ac2ff53 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:28.228101 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a0b228-31f9-43e8-84bf-18e37ac2ff53 None admin] PATH_INFO: `/v3/groups/da0669fc7bb94a6991321541b3f511e9` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:28.229926 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a0b228-31f9-43e8-84bf-18e37ac2ff53 None admin] RBAC: Authorizing `identity:get_group(group_id=da0669fc7bb94a6991321541b3f511e9)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:28.231578 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a0b228-31f9-43e8-84bf-18e37ac2ff53 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:28.235414 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 223/435] 10.222.0.27 () {56 vars in 1180 bytes} [Fri Aug 5 19:49:28 2022] GET /identity/v3/groups/da0669fc7bb94a6991321541b3f511e9 => generated 226 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:49:28.236949 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2164ca8-f6f5-4369-bce6-71b844cf6c98 None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:28.237408 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2164ca8-f6f5-4369-bce6-71b844cf6c98 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:28.237851 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2164ca8-f6f5-4369-bce6-71b844cf6c98 None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:28.239543 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 224/436] 10.222.0.27 () {54 vars in 927 bytes} [Fri Aug 5 19:49:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:49:28.240755 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2164ca8-f6f5-4369-bce6-71b844cf6c98 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:28.253691 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f113fc73-d6d4-49ce-b588-fd0bae8f9eba None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:28.296716 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ae6ef1-455f-4fa4-8eaf-5da7cb02c8d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:28.297282 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ae6ef1-455f-4fa4-8eaf-5da7cb02c8d1 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:28.297558 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ae6ef1-455f-4fa4-8eaf-5da7cb02c8d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:28.297828 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ae6ef1-455f-4fa4-8eaf-5da7cb02c8d1 None admin] PATH_INFO: `/v3/projects/6440bf75a9134de7ba2c5fb9c23955e2` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:28.299017 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ae6ef1-455f-4fa4-8eaf-5da7cb02c8d1 None admin] RBAC: Authorizing `identity:get_project(project_id=6440bf75a9134de7ba2c5fb9c23955e2)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:28.300480 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ae6ef1-455f-4fa4-8eaf-5da7cb02c8d1 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:28.302238 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 213/437] 10.222.0.27 () {56 vars in 1186 bytes} [Fri Aug 5 19:49:28 2022] GET /identity/v3/projects/6440bf75a9134de7ba2c5fb9c23955e2 => generated 302 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 19:49:28.303402 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ed052f-63f3-4111-9414-221d83abbea0 None None] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:28.303699 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ed052f-63f3-4111-9414-221d83abbea0 None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:28.304106 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ed052f-63f3-4111-9414-221d83abbea0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:28.325481 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.auth.core [None req-63ed052f-63f3-4111-9414-221d83abbea0 None None] MFA Rules not processed for user `03d1834a64cd4a43829827cb99a72747`. Rule list: `[]` (Enabled: `True`). {{(pid=80364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:49:28.326680 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-63ed052f-63f3-4111-9414-221d83abbea0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:49:28.393697 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2164ca8-f6f5-4369-bce6-71b844cf6c98 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:28.418285 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 214/438] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Aug 5 19:49:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 19:49:28.419459 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ed052f-63f3-4111-9414-221d83abbea0 None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:28.440768 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23793e27-3f34-45b3-86f0-17865dda6f74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:28.441884 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23793e27-3f34-45b3-86f0-17865dda6f74 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:28.442358 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23793e27-3f34-45b3-86f0-17865dda6f74 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:28.442641 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23793e27-3f34-45b3-86f0-17865dda6f74 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:28.443455 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23793e27-3f34-45b3-86f0-17865dda6f74 None admin] RBAC: Adding query filter params (group.id=da0669fc7bb94a6991321541b3f511e9, role.id=dde0aa8028794132913bcba4d91858b8, scope.project.id=6f67d76798fb469c8ec9dd2d185572c3) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:49:28.443772 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23793e27-3f34-45b3-86f0-17865dda6f74 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:28.445267 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23793e27-3f34-45b3-86f0-17865dda6f74 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:28.451355 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 225/439] 10.222.0.27 () {56 vars in 1376 bytes} [Fri Aug 5 19:49:28 2022] GET /identity/v3/role_assignments?group.id=da0669fc7bb94a6991321541b3f511e9&scope.project.id=6f67d76798fb469c8ec9dd2d185572c3&role.id=dde0aa8028794132913bcba4d91858b8 => generated 611 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 19:49:28.452155 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23793e27-3f34-45b3-86f0-17865dda6f74 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:28.490703 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:28.527285 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:28.576159 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23793e27-3f34-45b3-86f0-17865dda6f74 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:28.585395 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ed052f-63f3-4111-9414-221d83abbea0 None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:28.629219 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87be96c8-341c-4977-9613-ac082a114eb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:28.629889 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87be96c8-341c-4977-9613-ac082a114eb1 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:28.630175 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87be96c8-341c-4977-9613-ac082a114eb1 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:28.630448 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87be96c8-341c-4977-9613-ac082a114eb1 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:28.637847 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f178d3c8-e70d-47a7-a0af-da5dcf58bf1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:28.638518 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f178d3c8-e70d-47a7-a0af-da5dcf58bf1a None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:28.638801 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f178d3c8-e70d-47a7-a0af-da5dcf58bf1a None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:28.639096 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f178d3c8-e70d-47a7-a0af-da5dcf58bf1a None admin] PATH_INFO: `/v3/groups/da0669fc7bb94a6991321541b3f511e9` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:28.640273 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f178d3c8-e70d-47a7-a0af-da5dcf58bf1a None admin] RBAC: Authorizing `identity:get_group(group_id=da0669fc7bb94a6991321541b3f511e9)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:28.644183 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f178d3c8-e70d-47a7-a0af-da5dcf58bf1a None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:28.645811 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87be96c8-341c-4977-9613-ac082a114eb1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 19:49:28.646044 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 215/440] 10.222.0.27 () {56 vars in 1180 bytes} [Fri Aug 5 19:49:28 2022] GET /identity/v3/groups/da0669fc7bb94a6991321541b3f511e9 => generated 226 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:49:28.652316 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f178d3c8-e70d-47a7-a0af-da5dcf58bf1a None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:28.653757 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-87be96c8-341c-4977-9613-ac082a114eb1 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 05 19:49:28.653757 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:49:28.653757 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:49:28.653757 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:49:28.653757 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:49:28.653757 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:49:28.653757 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:49:28.653757 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:49:28.653757 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:49:28.653757 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:49:28.653757 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:49:28.653757 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:49:28.653757 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 19:49:28.653757 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 19:49:28.653757 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 19:49:28.653757 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 19:49:28.654687 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 19:49:28.654687 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 19:49:28.654687 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 19:49:28.654687 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 19:49:28.654687 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:49:28.654687 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:49:28.654687 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:49:28.654687 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:49:28.654687 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:49:28.654687 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:49:28.654687 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:49:28.654687 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Aug 05 19:49:28.654687 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:49:28.654687 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:49:28.654687 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:49:28.654687 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:49:28.656022 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:49:28.656022 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:49:28.656022 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:49:28.656022 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:49:28.656022 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 19:49:28.656022 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 19:49:28.656022 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 19:49:28.656022 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 19:49:28.656022 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 19:49:28.656022 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 19:49:28.656022 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 05 19:49:28.656022 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Aug 05 19:49:28.656663 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 226/441] 10.222.0.27 () {56 vars in 1114 bytes} [Fri Aug 5 19:49:28 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:49:28.663204 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87be96c8-341c-4977-9613-ac082a114eb1 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:28.793718 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87be96c8-341c-4977-9613-ac082a114eb1 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:28.799583 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f178d3c8-e70d-47a7-a0af-da5dcf58bf1a None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:28.811599 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:28.821822 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:28.837237 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9ef068-082a-4795-9c05-9dbace7c325b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:28.837789 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9ef068-082a-4795-9c05-9dbace7c325b None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:28.838073 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9ef068-082a-4795-9c05-9dbace7c325b None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:28.838330 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9ef068-082a-4795-9c05-9dbace7c325b None admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:28.839062 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9ef068-082a-4795-9c05-9dbace7c325b None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:49:28.839361 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9ef068-082a-4795-9c05-9dbace7c325b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:28.840753 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9ef068-082a-4795-9c05-9dbace7c325b None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:28.846646 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae3d8e2-3072-43e7-8c3c-46bbc66f4637 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:28.847462 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae3d8e2-3072-43e7-8c3c-46bbc66f4637 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:28.847748 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae3d8e2-3072-43e7-8c3c-46bbc66f4637 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:28.848029 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae3d8e2-3072-43e7-8c3c-46bbc66f4637 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:28.848871 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae3d8e2-3072-43e7-8c3c-46bbc66f4637 None admin] RBAC: Adding query filter params (group.id=da0669fc7bb94a6991321541b3f511e9, role.id=1a4f694d53054a24b580138b1ba77f3c, scope.project.id=6440bf75a9134de7ba2c5fb9c23955e2) {{(pid=80364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:49:28.849180 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae3d8e2-3072-43e7-8c3c-46bbc66f4637 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:28.850501 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 227/442] 10.222.0.27 () {56 vars in 1111 bytes} [Fri Aug 5 19:49:28 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 19:49:28.851716 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae3d8e2-3072-43e7-8c3c-46bbc66f4637 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:28.851820 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51eb9638-42f9-47c4-9e0d-b4a4b6f46f05 None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:28.852019 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51eb9638-42f9-47c4-9e0d-b4a4b6f46f05 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:28.852295 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51eb9638-42f9-47c4-9e0d-b4a4b6f46f05 None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:28.857667 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 228/443] 10.222.0.27 () {54 vars in 927 bytes} [Fri Aug 5 19:49:28 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:49:28.859645 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51eb9638-42f9-47c4-9e0d-b4a4b6f46f05 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:28.863996 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 216/444] 10.222.0.27 () {56 vars in 1376 bytes} [Fri Aug 5 19:49:28 2022] GET /identity/v3/role_assignments?group.id=da0669fc7bb94a6991321541b3f511e9&scope.project.id=6440bf75a9134de7ba2c5fb9c23955e2&role.id=1a4f694d53054a24b580138b1ba77f3c => generated 611 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 19:49:28.865301 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6d3288-7ede-4b0b-9f3a-720aec41d2a1 None None] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:28.865582 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6d3288-7ede-4b0b-9f3a-720aec41d2a1 None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:28.865855 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6d3288-7ede-4b0b-9f3a-720aec41d2a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:28.885394 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.auth.core [None req-0a6d3288-7ede-4b0b-9f3a-720aec41d2a1 None None] MFA Rules not processed for user `03d1834a64cd4a43829827cb99a72747`. Rule list: `[]` (Enabled: `True`). {{(pid=80364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:49:28.886620 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-0a6d3288-7ede-4b0b-9f3a-720aec41d2a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:49:28.962446 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 217/445] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Aug 5 19:49:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 19:49:28.982963 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51eb9638-42f9-47c4-9e0d-b4a4b6f46f05 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:28.986009 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6d3288-7ede-4b0b-9f3a-720aec41d2a1 None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:29.022947 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9827b5e4-cdbb-4935-b350-1499c315d384 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:29.023572 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9827b5e4-cdbb-4935-b350-1499c315d384 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:29.023855 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9827b5e4-cdbb-4935-b350-1499c315d384 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:29.024131 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9827b5e4-cdbb-4935-b350-1499c315d384 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:29.085011 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:29.104782 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6d3288-7ede-4b0b-9f3a-720aec41d2a1 None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:29.106094 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:29.107619 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9827b5e4-cdbb-4935-b350-1499c315d384 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:29.109827 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9827b5e4-cdbb-4935-b350-1499c315d384 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:29.147505 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f92961c-3126-4f88-8745-0b10c94da5ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:29.148154 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f92961c-3126-4f88-8745-0b10c94da5ed None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:29.148432 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f92961c-3126-4f88-8745-0b10c94da5ed None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:29.148736 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f92961c-3126-4f88-8745-0b10c94da5ed None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:29.162352 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f92961c-3126-4f88-8745-0b10c94da5ed None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 19:49:29.167438 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application [None req-6f92961c-3126-4f88-8745-0b10c94da5ed None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 19:49:29.167438 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:49:29.167438 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:49:29.167438 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:49:29.167438 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:49:29.167438 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:49:29.167438 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:49:29.167438 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:49:29.167438 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:49:29.167438 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:49:29.167438 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:49:29.167438 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:49:29.167438 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 19:49:29.167438 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 19:49:29.167438 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 19:49:29.167438 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 19:49:29.168315 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 19:49:29.168315 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 19:49:29.168315 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 19:49:29.168315 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 19:49:29.168315 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:49:29.168315 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:49:29.168315 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:49:29.168315 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:49:29.168315 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:49:29.168315 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:49:29.168315 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:49:29.168315 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application with Lock( Aug 05 19:49:29.168315 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:49:29.168315 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:49:29.168315 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:49:29.168315 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:49:29.169159 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:49:29.169159 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:49:29.169159 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:49:29.169159 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:49:29.169159 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 19:49:29.169159 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 19:49:29.169159 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 19:49:29.169159 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 19:49:29.169159 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 19:49:29.169159 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 19:49:29.169159 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 19:49:29.169159 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Aug 05 19:49:29.169159 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 218/446] 10.222.0.27 () {56 vars in 1096 bytes} [Fri Aug 5 19:49:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 05 19:49:29.176572 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f92961c-3126-4f88-8745-0b10c94da5ed None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:29.221627 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 229/447] 10.222.0.27 () {58 vars in 1282 bytes} [Fri Aug 5 19:49:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 05 19:49:29.229009 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9827b5e4-cdbb-4935-b350-1499c315d384 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:29.289776 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f92961c-3126-4f88-8745-0b10c94da5ed None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:29.317947 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:29.323437 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702548ac-ffbf-4abf-b9cc-e2b987f7f2b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:29.324056 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702548ac-ffbf-4abf-b9cc-e2b987f7f2b3 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:29.324341 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702548ac-ffbf-4abf-b9cc-e2b987f7f2b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:29.324641 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702548ac-ffbf-4abf-b9cc-e2b987f7f2b3 None admin] PATH_INFO: `/v3/roles` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:29.325362 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-702548ac-ffbf-4abf-b9cc-e2b987f7f2b3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:49:29.325671 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-702548ac-ffbf-4abf-b9cc-e2b987f7f2b3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:29.327043 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-702548ac-ffbf-4abf-b9cc-e2b987f7f2b3 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:29.334319 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9827b5e4-cdbb-4935-b350-1499c315d384 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:29.335321 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 219/448] 10.222.0.27 () {56 vars in 1099 bytes} [Fri Aug 5 19:49:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:49:29.340824 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702548ac-ffbf-4abf-b9cc-e2b987f7f2b3 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:29.371441 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:29.375215 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b662e11-ca51-4c3b-a474-40325b8f48ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:29.375954 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b662e11-ca51-4c3b-a474-40325b8f48ae None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:29.376239 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b662e11-ca51-4c3b-a474-40325b8f48ae None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:29.376532 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b662e11-ca51-4c3b-a474-40325b8f48ae None admin] PATH_INFO: `/v3/projects/6440bf75a9134de7ba2c5fb9c23955e2` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:29.377708 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b662e11-ca51-4c3b-a474-40325b8f48ae None admin] RBAC: Authorizing `identity:get_project(project_id=6440bf75a9134de7ba2c5fb9c23955e2)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:29.379147 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b662e11-ca51-4c3b-a474-40325b8f48ae None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:29.380846 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 230/449] 10.222.0.27 () {56 vars in 1186 bytes} [Fri Aug 5 19:49:29 2022] GET /identity/v3/projects/6440bf75a9134de7ba2c5fb9c23955e2 => generated 302 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 19:49:29.386324 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b662e11-ca51-4c3b-a474-40325b8f48ae None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:29.443527 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702548ac-ffbf-4abf-b9cc-e2b987f7f2b3 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:29.478535 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83db4358-28ed-41fc-8d3e-3c396c0a83af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:29.479142 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83db4358-28ed-41fc-8d3e-3c396c0a83af None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:29.479422 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83db4358-28ed-41fc-8d3e-3c396c0a83af None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:29.479690 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83db4358-28ed-41fc-8d3e-3c396c0a83af None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:29.486238 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b662e11-ca51-4c3b-a474-40325b8f48ae None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:29.519551 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184b32a8-05f9-4175-93f1-f6f239e29410 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:29.520180 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184b32a8-05f9-4175-93f1-f6f239e29410 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:29.520454 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184b32a8-05f9-4175-93f1-f6f239e29410 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:29.520749 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184b32a8-05f9-4175-93f1-f6f239e29410 None admin] PATH_INFO: `/v3/groups/da0669fc7bb94a6991321541b3f511e9` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:29.521887 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184b32a8-05f9-4175-93f1-f6f239e29410 None admin] RBAC: Authorizing `identity:get_group(group_id=da0669fc7bb94a6991321541b3f511e9)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:29.523305 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184b32a8-05f9-4175-93f1-f6f239e29410 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:29.529945 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 231/450] 10.222.0.27 () {56 vars in 1180 bytes} [Fri Aug 5 19:49:29 2022] GET /identity/v3/groups/da0669fc7bb94a6991321541b3f511e9 => generated 226 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 19:49:29.530880 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184b32a8-05f9-4175-93f1-f6f239e29410 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:29.547159 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83db4358-28ed-41fc-8d3e-3c396c0a83af None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:29.548668 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83db4358-28ed-41fc-8d3e-3c396c0a83af None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:29.551995 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:29.573989 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:29.630535 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184b32a8-05f9-4175-93f1-f6f239e29410 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:29.644557 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 220/451] 10.222.0.27 () {58 vars in 1282 bytes} [Fri Aug 5 19:49:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 05 19:49:29.650366 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83db4358-28ed-41fc-8d3e-3c396c0a83af None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:29.669106 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3b7659-da61-44c4-af7c-e8ba5edce0e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:29.669710 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3b7659-da61-44c4-af7c-e8ba5edce0e8 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:29.669988 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3b7659-da61-44c4-af7c-e8ba5edce0e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:29.670260 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3b7659-da61-44c4-af7c-e8ba5edce0e8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:29.671081 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d3b7659-da61-44c4-af7c-e8ba5edce0e8 None admin] RBAC: Adding query filter params (group.id=da0669fc7bb94a6991321541b3f511e9, role.id=dde0aa8028794132913bcba4d91858b8, scope.project.id=6440bf75a9134de7ba2c5fb9c23955e2) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:49:29.671392 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d3b7659-da61-44c4-af7c-e8ba5edce0e8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:29.672858 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d3b7659-da61-44c4-af7c-e8ba5edce0e8 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:29.678433 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 232/452] 10.222.0.27 () {56 vars in 1376 bytes} [Fri Aug 5 19:49:29 2022] GET /identity/v3/role_assignments?group.id=da0669fc7bb94a6991321541b3f511e9&scope.project.id=6440bf75a9134de7ba2c5fb9c23955e2&role.id=dde0aa8028794132913bcba4d91858b8 => generated 611 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 19:49:29.765482 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83db4358-28ed-41fc-8d3e-3c396c0a83af None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:29.793004 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:29.803359 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40f516d-e6a3-4fdf-a075-36606b0f2bec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:29.804006 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40f516d-e6a3-4fdf-a075-36606b0f2bec None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:29.804282 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40f516d-e6a3-4fdf-a075-36606b0f2bec None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:29.804572 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40f516d-e6a3-4fdf-a075-36606b0f2bec None admin] PATH_INFO: `/v3/projects/d25cd7afe64843529ae31496a857a541` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:29.805766 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b40f516d-e6a3-4fdf-a075-36606b0f2bec None admin] RBAC: Authorizing `identity:get_project(project_id=d25cd7afe64843529ae31496a857a541)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:29.807218 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b40f516d-e6a3-4fdf-a075-36606b0f2bec None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:29.808952 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 221/453] 10.222.0.27 () {56 vars in 1186 bytes} [Fri Aug 5 19:49:29 2022] GET /identity/v3/projects/d25cd7afe64843529ae31496a857a541 => generated 344 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 19:49:29.814632 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3b7659-da61-44c4-af7c-e8ba5edce0e8 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:29.903394 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:29.920484 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3b7659-da61-44c4-af7c-e8ba5edce0e8 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:29.954646 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280e34c5-2043-4f1d-8b0d-021e926954f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:29.955278 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280e34c5-2043-4f1d-8b0d-021e926954f1 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:29.955557 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280e34c5-2043-4f1d-8b0d-021e926954f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:29.955826 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280e34c5-2043-4f1d-8b0d-021e926954f1 None admin] PATH_INFO: `/v3/groups/291297a2f54e403d911b8e326c3201be` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:29.956998 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280e34c5-2043-4f1d-8b0d-021e926954f1 None admin] RBAC: Authorizing `identity:get_group(group_id=291297a2f54e403d911b8e326c3201be)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:29.958435 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280e34c5-2043-4f1d-8b0d-021e926954f1 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:29.960010 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 233/454] 10.222.0.27 () {56 vars in 1180 bytes} [Fri Aug 5 19:49:29 2022] GET /identity/v3/groups/291297a2f54e403d911b8e326c3201be => generated 229 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 19:49:29.965990 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40f516d-e6a3-4fdf-a075-36606b0f2bec None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:30.066372 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40f516d-e6a3-4fdf-a075-36606b0f2bec None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:30.098562 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472847da-cd6f-499c-839f-0d011ad95dca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:30.099163 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472847da-cd6f-499c-839f-0d011ad95dca None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:30.099442 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472847da-cd6f-499c-839f-0d011ad95dca None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:30.099714 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472847da-cd6f-499c-839f-0d011ad95dca None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:30.100516 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-472847da-cd6f-499c-839f-0d011ad95dca None admin] RBAC: Adding query filter params (group.id=291297a2f54e403d911b8e326c3201be, role.id=c173688a960b46e5be65f0ccd6fbd734, scope.project.id=d25cd7afe64843529ae31496a857a541) {{(pid=80364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:49:30.100830 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-472847da-cd6f-499c-839f-0d011ad95dca None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:30.102277 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-472847da-cd6f-499c-839f-0d011ad95dca None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:30.107832 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 222/455] 10.222.0.27 () {56 vars in 1376 bytes} [Fri Aug 5 19:49:29 2022] GET /identity/v3/role_assignments?group.id=291297a2f54e403d911b8e326c3201be&scope.project.id=d25cd7afe64843529ae31496a857a541&role.id=c173688a960b46e5be65f0ccd6fbd734 => generated 611 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:49:32.544571 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f02d21-46c5-4bcc-8699-796b7eb10da9 None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:32.545585 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f02d21-46c5-4bcc-8699-796b7eb10da9 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:32.546600 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f02d21-46c5-4bcc-8699-796b7eb10da9 None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:32.548471 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 234/456] 10.222.0.27 () {54 vars in 927 bytes} [Fri Aug 5 19:49:32 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:49:32.554153 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cade412d-8c8f-42fb-a207-d72a72b92b60 None None] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:32.555000 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cade412d-8c8f-42fb-a207-d72a72b92b60 None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:32.555478 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cade412d-8c8f-42fb-a207-d72a72b92b60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:32.585280 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.auth.core [None req-cade412d-8c8f-42fb-a207-d72a72b92b60 None None] MFA Rules not processed for user `03d1834a64cd4a43829827cb99a72747`. Rule list: `[]` (Enabled: `True`). {{(pid=80364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:49:32.586811 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-cade412d-8c8f-42fb-a207-d72a72b92b60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:49:32.654011 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:32.670714 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 223/457] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Aug 5 19:49:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 19:49:32.704291 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f02d21-46c5-4bcc-8699-796b7eb10da9 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:32.789301 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b044e4e-0d5b-4cbe-ac41-d3594123e81a None None] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:32.789822 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b044e4e-0d5b-4cbe-ac41-d3594123e81a None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:32.790093 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b044e4e-0d5b-4cbe-ac41-d3594123e81a None None] PATH_INFO: `/` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:32.792784 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 224/458] 10.222.0.27 () {54 vars in 927 bytes} [Fri Aug 5 19:49:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:49:32.797168 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc493818-fe9e-4e39-9a0b-e89883a7fcbe None None] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:32.797640 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc493818-fe9e-4e39-9a0b-e89883a7fcbe None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:32.798051 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc493818-fe9e-4e39-9a0b-e89883a7fcbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:32.823909 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.auth.core [None req-dc493818-fe9e-4e39-9a0b-e89883a7fcbe None None] MFA Rules not processed for user `03d1834a64cd4a43829827cb99a72747`. Rule list: `[]` (Enabled: `True`). {{(pid=80364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:49:32.825144 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-dc493818-fe9e-4e39-9a0b-e89883a7fcbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:49:32.847048 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f02d21-46c5-4bcc-8699-796b7eb10da9 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:32.911589 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c20c08-b8cf-4d9d-960f-c60e7561d0c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:32.912144 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c20c08-b8cf-4d9d-960f-c60e7561d0c9 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:32.912423 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c20c08-b8cf-4d9d-960f-c60e7561d0c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:32.912719 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c20c08-b8cf-4d9d-960f-c60e7561d0c9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:32.930673 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 225/459] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Aug 5 19:49:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 05 19:49:32.932694 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c20c08-b8cf-4d9d-960f-c60e7561d0c9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:32.934178 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c20c08-b8cf-4d9d-960f-c60e7561d0c9 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:32.949430 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-d8c20c08-b8cf-4d9d-960f-c60e7561d0c9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:49:32.949430 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:49:32.949430 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:49:32.949430 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:49:32.949430 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:49:32.949430 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:49:32.949430 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:49:32.949430 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:49:32.949430 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:49:32.949430 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:49:32.949430 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:49:32.949430 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:49:32.949430 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 19:49:32.949430 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 19:49:32.949430 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 19:49:32.950308 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 19:49:32.950308 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:49:32.950308 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:49:32.950308 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:49:32.950308 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:49:32.950308 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:49:32.950308 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:49:32.950308 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:49:32.950308 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Aug 05 19:49:32.950308 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:49:32.950308 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:49:32.950308 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:49:32.950308 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:49:32.950308 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:49:32.950308 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:49:32.950308 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:49:32.950308 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:49:32.951394 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 19:49:32.951394 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 19:49:32.951394 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:49:32.951394 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Aug 05 19:49:32.951394 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 235/460] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Aug 5 19:49:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:49:32.957959 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc493818-fe9e-4e39-9a0b-e89883a7fcbe None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:32.973453 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c20c08-b8cf-4d9d-960f-c60e7561d0c9 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:32.982953 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:33.089272 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc493818-fe9e-4e39-9a0b-e89883a7fcbe None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:33.089489 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c20c08-b8cf-4d9d-960f-c60e7561d0c9 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:33.110990 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:33.127809 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9399caed-4403-471f-9d54-10f2d8544f45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:33.128437 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9399caed-4403-471f-9d54-10f2d8544f45 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:33.129060 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9399caed-4403-471f-9d54-10f2d8544f45 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:33.129423 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9399caed-4403-471f-9d54-10f2d8544f45 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:33.132157 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81bf712-67d7-472a-a4ec-80239eda7d8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:33.132959 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81bf712-67d7-472a-a4ec-80239eda7d8f None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:33.133379 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81bf712-67d7-472a-a4ec-80239eda7d8f None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:33.133809 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81bf712-67d7-472a-a4ec-80239eda7d8f None admin] PATH_INFO: `/v3/domains` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:33.134644 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d81bf712-67d7-472a-a4ec-80239eda7d8f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:49:33.135088 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d81bf712-67d7-472a-a4ec-80239eda7d8f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:33.136715 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d81bf712-67d7-472a-a4ec-80239eda7d8f None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:33.140538 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9399caed-4403-471f-9d54-10f2d8544f45 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:33.141995 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9399caed-4403-471f-9d54-10f2d8544f45 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:33.150529 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 226/461] 10.222.0.27 () {56 vars in 1109 bytes} [Fri Aug 5 19:49:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:49:33.152028 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3038e08-24cf-4c21-b278-066b056f3be1 None None] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:33.152459 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3038e08-24cf-4c21-b278-066b056f3be1 None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:33.152927 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3038e08-24cf-4c21-b278-066b056f3be1 None None] PATH_INFO: `/` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:33.154552 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 227/462] 10.222.0.27 () {54 vars in 927 bytes} [Fri Aug 5 19:49:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:49:33.155903 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3038e08-24cf-4c21-b278-066b056f3be1 None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:33.162400 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-9399caed-4403-471f-9d54-10f2d8544f45 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:49:33.162400 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:49:33.162400 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:49:33.162400 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:49:33.162400 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:49:33.162400 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:49:33.162400 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:49:33.162400 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:49:33.162400 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:49:33.162400 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:49:33.162400 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:49:33.162400 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:49:33.162400 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 19:49:33.162400 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 19:49:33.162400 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 19:49:33.163398 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 19:49:33.163398 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:49:33.163398 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:49:33.163398 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:49:33.163398 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:49:33.163398 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:49:33.163398 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:49:33.163398 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:49:33.163398 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Aug 05 19:49:33.163398 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:49:33.163398 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:49:33.163398 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:49:33.163398 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:49:33.163398 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:49:33.163398 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:49:33.163398 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:49:33.163398 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:49:33.164453 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 19:49:33.164453 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 19:49:33.164453 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:49:33.164453 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Aug 05 19:49:33.164453 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 236/463] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Aug 5 19:49:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:49:33.165053 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7c6590-b2d4-49bd-941e-1ba781299247 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:33.165341 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7c6590-b2d4-49bd-941e-1ba781299247 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:33.165621 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7c6590-b2d4-49bd-941e-1ba781299247 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:33.184319 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-5a7c6590-b2d4-49bd-941e-1ba781299247 None None] MFA Rules not processed for user `03d1834a64cd4a43829827cb99a72747`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:49:33.185639 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-5a7c6590-b2d4-49bd-941e-1ba781299247 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:49:33.240327 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:33.263059 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 237/464] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Aug 5 19:49:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 19:49:33.269539 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7c6590-b2d4-49bd-941e-1ba781299247 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:33.282289 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3038e08-24cf-4c21-b278-066b056f3be1 None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:33.320683 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1abba82-8e60-4892-b869-52220b1b57d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:33.321296 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1abba82-8e60-4892-b869-52220b1b57d6 None admin] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:33.321576 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1abba82-8e60-4892-b869-52220b1b57d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:33.321847 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1abba82-8e60-4892-b869-52220b1b57d6 None admin] PATH_INFO: `/v3/users` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:33.322516 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1abba82-8e60-4892-b869-52220b1b57d6 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:33.323950 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1abba82-8e60-4892-b869-52220b1b57d6 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:33.345965 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 228/465] 10.222.0.27 () {60 vars in 1132 bytes} [Fri Aug 5 19:49:33 2022] POST /identity/v3/users => generated 246 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 19:49:33.359423 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26461aca-ed3c-4758-abad-e4d7d50d93d6 None None] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:33.359744 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26461aca-ed3c-4758-abad-e4d7d50d93d6 None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:33.359981 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26461aca-ed3c-4758-abad-e4d7d50d93d6 None None] PATH_INFO: `/` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:33.361988 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 229/466] 10.222.0.27 () {54 vars in 927 bytes} [Fri Aug 5 19:49:33 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:49:33.372592 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26461aca-ed3c-4758-abad-e4d7d50d93d6 None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:33.400748 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7c6590-b2d4-49bd-941e-1ba781299247 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:33.407799 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:33.456527 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8a5694-754e-4269-b817-1994abe25fec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:33.461339 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8a5694-754e-4269-b817-1994abe25fec None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:33.461799 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8a5694-754e-4269-b817-1994abe25fec None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:33.462199 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8a5694-754e-4269-b817-1994abe25fec None admin] PATH_INFO: `/v3/domains` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:33.463314 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c8a5694-754e-4269-b817-1994abe25fec None admin] RBAC: Adding query filter params (name=Default) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:49:33.463763 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c8a5694-754e-4269-b817-1994abe25fec None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:33.465541 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c8a5694-754e-4269-b817-1994abe25fec None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:33.486484 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26461aca-ed3c-4758-abad-e4d7d50d93d6 None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:33.494791 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 238/467] 10.222.0.27 () {56 vars in 1109 bytes} [Fri Aug 5 19:49:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:49:33.496454 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7be02a5-589e-4067-bdb2-45eabb5e628f None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:33.496899 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7be02a5-589e-4067-bdb2-45eabb5e628f None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:33.497330 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7be02a5-589e-4067-bdb2-45eabb5e628f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:33.521576 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-c7be02a5-589e-4067-bdb2-45eabb5e628f None None] MFA Rules not processed for user `03d1834a64cd4a43829827cb99a72747`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:49:33.523097 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-c7be02a5-589e-4067-bdb2-45eabb5e628f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:49:33.539690 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e221e5b-7602-4d33-8e02-f8bcfb8d3817 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:33.540380 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e221e5b-7602-4d33-8e02-f8bcfb8d3817 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:33.540694 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e221e5b-7602-4d33-8e02-f8bcfb8d3817 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:33.540966 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e221e5b-7602-4d33-8e02-f8bcfb8d3817 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:33.553298 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e221e5b-7602-4d33-8e02-f8bcfb8d3817 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:33.554932 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e221e5b-7602-4d33-8e02-f8bcfb8d3817 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:33.573680 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application [None req-1e221e5b-7602-4d33-8e02-f8bcfb8d3817 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:49:33.573680 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:49:33.573680 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:49:33.573680 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:49:33.573680 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:49:33.573680 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:49:33.573680 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:49:33.573680 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:49:33.573680 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:49:33.573680 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:49:33.573680 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:49:33.573680 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:49:33.573680 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 19:49:33.573680 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 19:49:33.573680 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 19:49:33.574603 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 19:49:33.574603 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:49:33.574603 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:49:33.574603 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:49:33.574603 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:49:33.574603 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:49:33.574603 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:49:33.574603 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:49:33.574603 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application with Lock( Aug 05 19:49:33.574603 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:49:33.574603 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:49:33.574603 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:49:33.574603 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:49:33.574603 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:49:33.574603 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:49:33.574603 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:49:33.574603 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:49:33.575441 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 19:49:33.575441 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 19:49:33.575441 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:49:33.575441 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Aug 05 19:49:33.575441 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 230/468] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Aug 5 19:49:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:49:33.576725 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e221e5b-7602-4d33-8e02-f8bcfb8d3817 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:33.585385 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:33.616626 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 239/469] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Aug 5 19:49:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 19:49:33.617859 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7be02a5-589e-4067-bdb2-45eabb5e628f None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:33.689618 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:33.707178 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e221e5b-7602-4d33-8e02-f8bcfb8d3817 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:33.746333 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7be02a5-589e-4067-bdb2-45eabb5e628f None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:33.747034 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae106cb-49cf-4f85-b69b-7e85c4a76c36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:33.747852 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae106cb-49cf-4f85-b69b-7e85c4a76c36 None admin] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:33.748188 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae106cb-49cf-4f85-b69b-7e85c4a76c36 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:33.748470 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae106cb-49cf-4f85-b69b-7e85c4a76c36 None admin] PATH_INFO: `/v3/users` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:33.750509 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae106cb-49cf-4f85-b69b-7e85c4a76c36 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:33.751986 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae106cb-49cf-4f85-b69b-7e85c4a76c36 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:33.783622 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 231/470] 10.222.0.27 () {60 vars in 1132 bytes} [Fri Aug 5 19:49:33 2022] POST /identity/v3/users => generated 248 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 19:49:33.783622 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae106cb-49cf-4f85-b69b-7e85c4a76c36 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:33.818590 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33732d4-26d5-4eae-bb78-d1bb324ad7e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:33.819476 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33732d4-26d5-4eae-bb78-d1bb324ad7e6 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:33.819907 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33732d4-26d5-4eae-bb78-d1bb324ad7e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:33.820343 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33732d4-26d5-4eae-bb78-d1bb324ad7e6 None admin] PATH_INFO: `/v3/domains` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:33.821739 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d33732d4-26d5-4eae-bb78-d1bb324ad7e6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:49:33.822196 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d33732d4-26d5-4eae-bb78-d1bb324ad7e6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:33.824174 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d33732d4-26d5-4eae-bb78-d1bb324ad7e6 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:33.856408 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 240/471] 10.222.0.27 () {56 vars in 1109 bytes} [Fri Aug 5 19:49:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:49:33.862041 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33732d4-26d5-4eae-bb78-d1bb324ad7e6 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:33.933469 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:33.999669 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae106cb-49cf-4f85-b69b-7e85c4a76c36 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:34.058636 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da2c11a-20c5-46db-96b6-7b48816a48f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:34.059286 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da2c11a-20c5-46db-96b6-7b48816a48f7 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:34.059566 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da2c11a-20c5-46db-96b6-7b48816a48f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:34.059836 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da2c11a-20c5-46db-96b6-7b48816a48f7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:34.078794 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33732d4-26d5-4eae-bb78-d1bb324ad7e6 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:34.081102 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da2c11a-20c5-46db-96b6-7b48816a48f7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:34.082780 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da2c11a-20c5-46db-96b6-7b48816a48f7 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:34.092873 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:34.106144 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application [None req-6da2c11a-20c5-46db-96b6-7b48816a48f7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:49:34.106144 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:49:34.106144 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:49:34.106144 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:49:34.106144 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:49:34.106144 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:49:34.106144 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:49:34.106144 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:49:34.106144 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:49:34.106144 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:49:34.106144 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:49:34.106144 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:49:34.106144 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 19:49:34.106144 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 19:49:34.106144 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 19:49:34.107033 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 19:49:34.107033 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:49:34.107033 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:49:34.107033 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:49:34.107033 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:49:34.107033 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:49:34.107033 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:49:34.107033 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:49:34.107033 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application with Lock( Aug 05 19:49:34.107033 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:49:34.107033 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:49:34.107033 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:49:34.107033 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:49:34.107033 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:49:34.107033 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:49:34.107033 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:49:34.107033 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:49:34.107897 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 19:49:34.107897 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 19:49:34.107897 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:49:34.107897 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Aug 05 19:49:34.107897 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 232/472] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Aug 5 19:49:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 327 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:49:34.109291 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f48277-94b5-4bb0-96da-5945a0a481a1 None None] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:34.109625 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f48277-94b5-4bb0-96da-5945a0a481a1 None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:34.109906 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f48277-94b5-4bb0-96da-5945a0a481a1 None None] PATH_INFO: `/` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:34.111403 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 233/473] 10.222.0.27 () {54 vars in 927 bytes} [Fri Aug 5 19:49:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:49:34.117051 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdb6eae-b8b9-4bba-9e37-951750104f3f None None] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:34.117520 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdb6eae-b8b9-4bba-9e37-951750104f3f None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:34.117928 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdb6eae-b8b9-4bba-9e37-951750104f3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:34.135144 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332104f4-049f-4355-b7d1-d71888ba61d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:34.135772 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332104f4-049f-4355-b7d1-d71888ba61d3 None admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:34.136057 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332104f4-049f-4355-b7d1-d71888ba61d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:34.136380 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332104f4-049f-4355-b7d1-d71888ba61d3 None admin] PATH_INFO: `/v3/users` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:34.137109 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-332104f4-049f-4355-b7d1-d71888ba61d3 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:34.138618 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-332104f4-049f-4355-b7d1-d71888ba61d3 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:34.145235 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.auth.core [None req-5bdb6eae-b8b9-4bba-9e37-951750104f3f None None] MFA Rules not processed for user `03d1834a64cd4a43829827cb99a72747`. Rule list: `[]` (Enabled: `True`). {{(pid=80364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:49:34.146734 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-5bdb6eae-b8b9-4bba-9e37-951750104f3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:49:34.163658 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 241/474] 10.222.0.27 () {60 vars in 1132 bytes} [Fri Aug 5 19:49:33 2022] POST /identity/v3/users => generated 248 bytes in 302 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 19:49:34.165072 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332104f4-049f-4355-b7d1-d71888ba61d3 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:34.297570 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 234/475] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Aug 5 19:49:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 05 19:49:34.321265 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bdb6eae-b8b9-4bba-9e37-951750104f3f None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:34.368203 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:34.382426 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332104f4-049f-4355-b7d1-d71888ba61d3 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:34.426298 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342065e2-7174-4594-870f-ec6d59e9a9c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:34.426920 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342065e2-7174-4594-870f-ec6d59e9a9c7 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:34.427200 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342065e2-7174-4594-870f-ec6d59e9a9c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:34.427476 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342065e2-7174-4594-870f-ec6d59e9a9c7 None admin] PATH_INFO: `/v3/domains` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:34.428181 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-342065e2-7174-4594-870f-ec6d59e9a9c7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:49:34.428486 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-342065e2-7174-4594-870f-ec6d59e9a9c7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:34.430835 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-342065e2-7174-4594-870f-ec6d59e9a9c7 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:34.444917 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 242/476] 10.222.0.27 () {56 vars in 1109 bytes} [Fri Aug 5 19:49:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:49:34.450160 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342065e2-7174-4594-870f-ec6d59e9a9c7 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:34.468850 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bdb6eae-b8b9-4bba-9e37-951750104f3f None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:34.497524 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:34.512611 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5bcdf9-fb29-4e50-801b-bc1e19e2b86f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:34.513572 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5bcdf9-fb29-4e50-801b-bc1e19e2b86f None admin] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:34.513923 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5bcdf9-fb29-4e50-801b-bc1e19e2b86f None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:34.514652 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5bcdf9-fb29-4e50-801b-bc1e19e2b86f None admin] PATH_INFO: `/v3/roles` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:34.515402 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a5bcdf9-fb29-4e50-801b-bc1e19e2b86f None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:34.517320 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a5bcdf9-fb29-4e50-801b-bc1e19e2b86f None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:34.526814 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.sql.core [None req-7a5bcdf9-fb29-4e50-801b-bc1e19e2b86f None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 05 19:49:34.526814 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Aug 05 19:49:34.526814 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [parameters: {'id': 'ae47846fd9f44b5a9e2fa479e716ca83', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Aug 05 19:49:34.526814 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80364) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 05 19:49:34.549563 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application [None req-7a5bcdf9-fb29-4e50-801b-bc1e19e2b86f None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Aug 05 19:49:34.549563 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:49:34.549563 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 05 19:49:34.549563 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 05 19:49:34.549563 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 05 19:49:34.549563 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 05 19:49:34.549563 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 05 19:49:34.549563 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application result = self._query(query) Aug 05 19:49:34.549563 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 05 19:49:34.549563 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application conn.query(q) Aug 05 19:49:34.549563 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 05 19:49:34.549563 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 05 19:49:34.549563 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 05 19:49:34.549563 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application result.read() Aug 05 19:49:34.549563 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 05 19:49:34.550436 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 05 19:49:34.550436 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 05 19:49:34.550436 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 05 19:49:34.550436 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 05 19:49:34.550436 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 05 19:49:34.550436 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 05 19:49:34.550436 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 05 19:49:34.550436 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 05 19:49:34.550436 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Aug 05 19:49:34.550436 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 05 19:49:34.550436 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Aug 05 19:49:34.550436 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:49:34.550436 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 05 19:49:34.550436 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 05 19:49:34.550436 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Aug 05 19:49:34.550436 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return ref.to_dict() Aug 05 19:49:34.550436 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 05 19:49:34.550436 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application next(self.gen) Aug 05 19:49:34.551324 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 05 19:49:34.551324 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application yield resource Aug 05 19:49:34.551324 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 05 19:49:34.551324 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application next(self.gen) Aug 05 19:49:34.551324 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 05 19:49:34.551324 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application self.session.rollback() Aug 05 19:49:34.551324 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 05 19:49:34.551324 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application self.force_reraise() Aug 05 19:49:34.551324 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 05 19:49:34.551324 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application raise self.value Aug 05 19:49:34.551324 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 05 19:49:34.551324 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 05 19:49:34.551324 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 05 19:49:34.551324 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application session.commit() Aug 05 19:49:34.551324 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 05 19:49:34.551324 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 05 19:49:34.551324 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 05 19:49:34.552168 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application self._prepare_impl() Aug 05 19:49:34.552168 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 05 19:49:34.552168 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application self.session.flush() Aug 05 19:49:34.552168 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 05 19:49:34.552168 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application self._flush(objects) Aug 05 19:49:34.552168 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 05 19:49:34.552168 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 05 19:49:34.552168 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 05 19:49:34.552168 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application compat.raise_( Aug 05 19:49:34.552168 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 05 19:49:34.552168 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application raise exception Aug 05 19:49:34.552168 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 05 19:49:34.552168 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application flush_context.execute() Aug 05 19:49:34.552168 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 05 19:49:34.552168 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application rec.execute(self) Aug 05 19:49:34.552168 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 05 19:49:34.552168 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 05 19:49:34.554386 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 05 19:49:34.554386 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 05 19:49:34.554386 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Aug 05 19:49:34.554386 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application c = connection._execute_20( Aug 05 19:49:34.554386 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Aug 05 19:49:34.554386 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 05 19:49:34.554386 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Aug 05 19:49:34.554386 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 05 19:49:34.554386 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Aug 05 19:49:34.554386 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 05 19:49:34.554386 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Aug 05 19:49:34.554386 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 05 19:49:34.554386 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Aug 05 19:49:34.554386 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 05 19:49:34.554386 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 05 19:49:34.554386 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application raise exception Aug 05 19:49:34.556686 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 05 19:49:34.556686 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 05 19:49:34.556686 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 05 19:49:34.556686 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 05 19:49:34.556686 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 05 19:49:34.556686 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application result = self._query(query) Aug 05 19:49:34.556686 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 05 19:49:34.556686 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application conn.query(q) Aug 05 19:49:34.556686 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 05 19:49:34.556686 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 05 19:49:34.556686 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 05 19:49:34.556686 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application result.read() Aug 05 19:49:34.556686 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 05 19:49:34.556686 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 05 19:49:34.556686 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 05 19:49:34.556686 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 05 19:49:34.557646 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 05 19:49:34.557646 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 05 19:49:34.557646 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 05 19:49:34.557646 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 05 19:49:34.557646 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 05 19:49:34.557646 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Aug 05 19:49:34.557646 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application [parameters: {'id': 'ae47846fd9f44b5a9e2fa479e716ca83', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Aug 05 19:49:34.557646 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 05 19:49:34.557646 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Aug 05 19:49:34.557646 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 05 19:49:34.557646 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Aug 05 19:49:34.557646 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:49:34.557646 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:49:34.557646 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:49:34.557646 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:49:34.557646 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:49:34.558566 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:49:34.558566 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:49:34.558566 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:49:34.558566 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:49:34.558566 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:49:34.558566 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:49:34.558566 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Aug 05 19:49:34.558566 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Aug 05 19:49:34.558566 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:49:34.558566 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:49:34.558566 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Aug 05 19:49:34.558566 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Aug 05 19:49:34.558566 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 05 19:49:34.558566 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 05 19:49:34.558566 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Aug 05 19:49:34.558566 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Aug 05 19:49:34.569223 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 235/477] 10.222.0.27 () {60 vars in 1131 bytes} [Fri Aug 5 19:49:34 2022] POST /identity/v3/roles => generated 144 bytes in 230 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Aug 05 19:49:34.569223 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5bcdf9-fb29-4e50-801b-bc1e19e2b86f None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:34.572715 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342065e2-7174-4594-870f-ec6d59e9a9c7 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:34.620359 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f55dd7e-95f0-4063-a2e7-7fb024d697b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:34.621655 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f55dd7e-95f0-4063-a2e7-7fb024d697b3 None admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:34.622010 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f55dd7e-95f0-4063-a2e7-7fb024d697b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:34.622355 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f55dd7e-95f0-4063-a2e7-7fb024d697b3 None admin] PATH_INFO: `/v3/users` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:34.623536 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f55dd7e-95f0-4063-a2e7-7fb024d697b3 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:34.626172 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f55dd7e-95f0-4063-a2e7-7fb024d697b3 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:34.655061 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 243/478] 10.222.0.27 () {60 vars in 1132 bytes} [Fri Aug 5 19:49:34 2022] POST /identity/v3/users => generated 249 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 05 19:49:34.708006 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5bcdf9-fb29-4e50-801b-bc1e19e2b86f None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:34.759459 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11fdb66-a620-4d97-9b9b-1888581d84d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:34.760168 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11fdb66-a620-4d97-9b9b-1888581d84d8 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:34.760453 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11fdb66-a620-4d97-9b9b-1888581d84d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:34.760758 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11fdb66-a620-4d97-9b9b-1888581d84d8 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:34.773007 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c11fdb66-a620-4d97-9b9b-1888581d84d8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 19:49:34.778483 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application [None req-c11fdb66-a620-4d97-9b9b-1888581d84d8 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 05 19:49:34.778483 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:49:34.778483 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:49:34.778483 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:49:34.778483 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:49:34.778483 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:49:34.778483 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:49:34.778483 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:49:34.778483 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:49:34.778483 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:49:34.778483 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:49:34.778483 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:49:34.778483 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 19:49:34.778483 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 19:49:34.778483 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 19:49:34.778483 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 19:49:34.780235 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 19:49:34.780235 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 19:49:34.780235 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 19:49:34.780235 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 19:49:34.780235 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:49:34.780235 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:49:34.780235 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:49:34.780235 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:49:34.780235 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:49:34.780235 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:49:34.780235 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:49:34.780235 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application with Lock( Aug 05 19:49:34.780235 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:49:34.780235 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:49:34.780235 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:49:34.780235 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:49:34.781937 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:49:34.781937 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:49:34.781937 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:49:34.781937 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:49:34.781937 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 19:49:34.781937 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 19:49:34.781937 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 19:49:34.781937 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 19:49:34.781937 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 19:49:34.781937 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 19:49:34.781937 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 05 19:49:34.781937 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Aug 05 19:49:34.783198 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 236/479] 10.222.0.27 () {56 vars in 1114 bytes} [Fri Aug 5 19:49:34 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:49:34.786090 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f55dd7e-95f0-4063-a2e7-7fb024d697b3 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:34.808974 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:34.923130 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f55dd7e-95f0-4063-a2e7-7fb024d697b3 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:34.965967 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a3fc60-dfd7-4473-8c48-b7533b954651 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:34.966603 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a3fc60-dfd7-4473-8c48-b7533b954651 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:34.966885 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a3fc60-dfd7-4473-8c48-b7533b954651 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:34.967160 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a3fc60-dfd7-4473-8c48-b7533b954651 None admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:34.968579 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a3fc60-dfd7-4473-8c48-b7533b954651 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:49:34.968936 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a3fc60-dfd7-4473-8c48-b7533b954651 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:34.970350 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a3fc60-dfd7-4473-8c48-b7533b954651 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:34.982334 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 244/480] 10.222.0.27 () {56 vars in 1111 bytes} [Fri Aug 5 19:49:34 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:49:36.019282 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92042c1c-62eb-4fdd-a2ab-536561ea4713 None None] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:36.019977 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92042c1c-62eb-4fdd-a2ab-536561ea4713 None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:36.019977 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92042c1c-62eb-4fdd-a2ab-536561ea4713 None None] PATH_INFO: `/` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:36.021474 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 237/481] 10.222.0.27 () {54 vars in 927 bytes} [Fri Aug 5 19:49:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:49:36.026979 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8fd5d9-d932-45bd-b8a0-22fa233c23f0 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:36.027434 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8fd5d9-d932-45bd-b8a0-22fa233c23f0 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:36.027582 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8fd5d9-d932-45bd-b8a0-22fa233c23f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:36.052585 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-fa8fd5d9-d932-45bd-b8a0-22fa233c23f0 None None] MFA Rules not processed for user `03d1834a64cd4a43829827cb99a72747`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:49:36.053781 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-fa8fd5d9-d932-45bd-b8a0-22fa233c23f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:49:36.145822 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:36.151768 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 245/482] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Aug 5 19:49:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 19:49:36.175666 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92042c1c-62eb-4fdd-a2ab-536561ea4713 None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:36.223234 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb032fea-d404-4c76-89f8-3cdc31e6d1ee None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:36.223567 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb032fea-d404-4c76-89f8-3cdc31e6d1ee None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:36.223784 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb032fea-d404-4c76-89f8-3cdc31e6d1ee None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:36.225172 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 246/483] 10.222.0.27 () {54 vars in 927 bytes} [Fri Aug 5 19:49:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 05 19:49:36.230682 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4adb69-9b44-4175-bdd2-daa747091fa8 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:36.230968 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4adb69-9b44-4175-bdd2-daa747091fa8 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:36.231243 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4adb69-9b44-4175-bdd2-daa747091fa8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:36.237900 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:36.250127 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-9d4adb69-9b44-4175-bdd2-daa747091fa8 None None] MFA Rules not processed for user `03d1834a64cd4a43829827cb99a72747`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:49:36.251321 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-9d4adb69-9b44-4175-bdd2-daa747091fa8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:49:36.301341 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92042c1c-62eb-4fdd-a2ab-536561ea4713 None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:36.341104 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3466576c-ecfe-40d7-876c-0c500349df76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:36.342144 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3466576c-ecfe-40d7-876c-0c500349df76 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:36.342430 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3466576c-ecfe-40d7-876c-0c500349df76 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:36.342770 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3466576c-ecfe-40d7-876c-0c500349df76 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:36.355130 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3466576c-ecfe-40d7-876c-0c500349df76 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 19:49:36.371813 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application [None req-3466576c-ecfe-40d7-876c-0c500349df76 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 05 19:49:36.371813 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:49:36.371813 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:49:36.371813 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:49:36.371813 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:49:36.371813 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:49:36.371813 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:49:36.371813 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:49:36.371813 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:49:36.371813 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:49:36.371813 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:49:36.371813 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:49:36.371813 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 19:49:36.371813 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 19:49:36.371813 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 19:49:36.371813 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 19:49:36.373889 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 19:49:36.373889 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 19:49:36.373889 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 19:49:36.373889 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 19:49:36.373889 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:49:36.373889 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:49:36.373889 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:49:36.373889 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:49:36.373889 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:49:36.373889 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:49:36.373889 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:49:36.373889 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application with Lock( Aug 05 19:49:36.373889 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:49:36.373889 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:49:36.373889 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:49:36.373889 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:49:36.375801 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:49:36.375801 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:49:36.375801 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:49:36.375801 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:49:36.375801 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 19:49:36.375801 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 19:49:36.375801 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 19:49:36.375801 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 19:49:36.375801 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 19:49:36.375801 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 19:49:36.375801 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 05 19:49:36.375801 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Aug 05 19:49:36.375801 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 238/484] 10.222.0.27 () {56 vars in 1102 bytes} [Fri Aug 5 19:49:36 2022] GET /identity/v3/roles/service => generated 85 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:49:36.384524 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3466576c-ecfe-40d7-876c-0c500349df76 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:36.393656 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 247/485] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Aug 5 19:49:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 19:49:36.431145 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4adb69-9b44-4175-bdd2-daa747091fa8 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:36.494031 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3466576c-ecfe-40d7-876c-0c500349df76 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:36.505616 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:36.535969 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34314a85-64e8-4b9a-95f0-7af138b36578 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:36.536734 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34314a85-64e8-4b9a-95f0-7af138b36578 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:36.537088 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34314a85-64e8-4b9a-95f0-7af138b36578 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:36.537918 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34314a85-64e8-4b9a-95f0-7af138b36578 None admin] PATH_INFO: `/v3/roles` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:36.538854 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34314a85-64e8-4b9a-95f0-7af138b36578 None admin] RBAC: Adding query filter params (name=service) {{(pid=80364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:49:36.539369 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34314a85-64e8-4b9a-95f0-7af138b36578 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:36.541055 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34314a85-64e8-4b9a-95f0-7af138b36578 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:36.550784 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 239/486] 10.222.0.27 () {56 vars in 1103 bytes} [Fri Aug 5 19:49:36 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 19:49:36.556266 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34314a85-64e8-4b9a-95f0-7af138b36578 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:36.630843 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4adb69-9b44-4175-bdd2-daa747091fa8 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:36.661623 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34314a85-64e8-4b9a-95f0-7af138b36578 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:36.694656 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:36.699800 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a90f93d-e748-4ce8-8002-1641774f0cb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:36.700818 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a90f93d-e748-4ce8-8002-1641774f0cb4 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:36.701354 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a90f93d-e748-4ce8-8002-1641774f0cb4 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:36.701840 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a90f93d-e748-4ce8-8002-1641774f0cb4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:36.711207 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ed08a4-9185-4abd-b9ab-fcf6eba0d93c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:36.711566 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ed08a4-9185-4abd-b9ab-fcf6eba0d93c None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:36.711796 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ed08a4-9185-4abd-b9ab-fcf6eba0d93c None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:36.712071 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ed08a4-9185-4abd-b9ab-fcf6eba0d93c None admin] PATH_INFO: `/v3/roles/service` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:36.716607 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a90f93d-e748-4ce8-8002-1641774f0cb4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:36.719165 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a90f93d-e748-4ce8-8002-1641774f0cb4 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:36.735582 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application [None req-2a90f93d-e748-4ce8-8002-1641774f0cb4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:49:36.735582 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:49:36.735582 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:49:36.735582 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:49:36.735582 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:49:36.735582 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:49:36.735582 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:49:36.735582 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:49:36.735582 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:49:36.735582 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:49:36.735582 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:49:36.735582 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:49:36.735582 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 19:49:36.735582 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 19:49:36.735582 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 19:49:36.736481 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 19:49:36.736481 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:49:36.736481 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:49:36.736481 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:49:36.736481 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:49:36.736481 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:49:36.736481 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:49:36.736481 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:49:36.736481 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application with Lock( Aug 05 19:49:36.736481 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:49:36.736481 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:49:36.736481 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:49:36.736481 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:49:36.736481 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:49:36.736481 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:49:36.736481 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:49:36.736481 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:49:36.737352 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 19:49:36.737352 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 19:49:36.737352 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:49:36.737352 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Aug 05 19:49:36.739502 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 240/487] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Aug 5 19:49:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 05 19:49:36.744199 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ed08a4-9185-4abd-b9ab-fcf6eba0d93c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 19:49:36.745275 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a90f93d-e748-4ce8-8002-1641774f0cb4 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:36.746292 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-85ed08a4-9185-4abd-b9ab-fcf6eba0d93c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 05 19:49:36.746292 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:49:36.746292 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:49:36.746292 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:49:36.746292 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:49:36.746292 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:49:36.746292 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:49:36.746292 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:49:36.746292 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:49:36.746292 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:49:36.746292 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:49:36.746292 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:49:36.746292 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 19:49:36.746292 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 19:49:36.746292 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 19:49:36.746292 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 19:49:36.747144 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 19:49:36.747144 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 19:49:36.747144 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 19:49:36.747144 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 19:49:36.747144 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:49:36.747144 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:49:36.747144 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:49:36.747144 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:49:36.747144 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:49:36.747144 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:49:36.747144 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:49:36.747144 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Aug 05 19:49:36.747144 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:49:36.747144 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:49:36.747144 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:49:36.747144 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:49:36.747929 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:49:36.747929 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:49:36.747929 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:49:36.747929 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:49:36.747929 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 19:49:36.747929 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 19:49:36.747929 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 19:49:36.747929 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 19:49:36.747929 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 19:49:36.747929 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 19:49:36.747929 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 05 19:49:36.747929 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Aug 05 19:49:36.747929 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 248/488] 10.222.0.27 () {56 vars in 1102 bytes} [Fri Aug 5 19:49:36 2022] GET /identity/v3/roles/service => generated 85 bytes in 317 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 05 19:49:36.753579 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ed08a4-9185-4abd-b9ab-fcf6eba0d93c None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:36.804907 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:36.856206 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a90f93d-e748-4ce8-8002-1641774f0cb4 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:36.892213 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593abb55-dfe5-4bfa-9ad1-58fa5ccb0123 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:36.893091 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593abb55-dfe5-4bfa-9ad1-58fa5ccb0123 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:36.893701 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593abb55-dfe5-4bfa-9ad1-58fa5ccb0123 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:36.894131 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593abb55-dfe5-4bfa-9ad1-58fa5ccb0123 None admin] PATH_INFO: `/v3/domains` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:36.894975 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593abb55-dfe5-4bfa-9ad1-58fa5ccb0123 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:49:36.895488 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593abb55-dfe5-4bfa-9ad1-58fa5ccb0123 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:36.897070 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593abb55-dfe5-4bfa-9ad1-58fa5ccb0123 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:36.910707 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 241/489] 10.222.0.27 () {56 vars in 1109 bytes} [Fri Aug 5 19:49:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:49:36.916358 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593abb55-dfe5-4bfa-9ad1-58fa5ccb0123 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:36.942956 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ed08a4-9185-4abd-b9ab-fcf6eba0d93c None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:36.978739 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1c9061-c00d-47b7-ada1-743ae403f574 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:36.979306 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1c9061-c00d-47b7-ada1-743ae403f574 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:36.979565 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1c9061-c00d-47b7-ada1-743ae403f574 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:36.979843 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1c9061-c00d-47b7-ada1-743ae403f574 None admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:36.980608 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1c9061-c00d-47b7-ada1-743ae403f574 None admin] RBAC: Adding query filter params (name=service) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:49:36.980917 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1c9061-c00d-47b7-ada1-743ae403f574 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:36.982338 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1c9061-c00d-47b7-ada1-743ae403f574 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:36.990984 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 249/490] 10.222.0.27 () {56 vars in 1103 bytes} [Fri Aug 5 19:49:36 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:49:36.992314 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a2bca5-2cd2-47ae-8e9c-40fca10d49fd None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:36.992646 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a2bca5-2cd2-47ae-8e9c-40fca10d49fd None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:36.992946 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a2bca5-2cd2-47ae-8e9c-40fca10d49fd None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:36.996012 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 250/491] 10.222.0.27 () {54 vars in 927 bytes} [Fri Aug 5 19:49:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:49:36.996351 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a2bca5-2cd2-47ae-8e9c-40fca10d49fd None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:37.017962 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:37.033441 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593abb55-dfe5-4bfa-9ad1-58fa5ccb0123 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:37.058474 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:37.084910 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2108ab10-c400-465d-9ab5-4cf07426477c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:37.086337 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2108ab10-c400-465d-9ab5-4cf07426477c None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:37.086844 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2108ab10-c400-465d-9ab5-4cf07426477c None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:37.087868 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2108ab10-c400-465d-9ab5-4cf07426477c None admin] PATH_INFO: `/v3/users/nova` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:37.098265 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2108ab10-c400-465d-9ab5-4cf07426477c None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:37.100601 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2108ab10-c400-465d-9ab5-4cf07426477c None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:37.115692 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application [None req-2108ab10-c400-465d-9ab5-4cf07426477c None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 05 19:49:37.115692 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:49:37.115692 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:49:37.115692 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:49:37.115692 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:49:37.115692 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:49:37.115692 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:49:37.115692 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:49:37.115692 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:49:37.115692 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:49:37.115692 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:49:37.115692 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:49:37.115692 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 05 19:49:37.115692 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 05 19:49:37.115692 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 05 19:49:37.115692 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 05 19:49:37.116660 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:49:37.116660 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:49:37.116660 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 05 19:49:37.116660 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 19:49:37.116660 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 05 19:49:37.116660 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 19:49:37.116660 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:49:37.116660 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:49:37.116660 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:49:37.116660 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:49:37.116660 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:49:37.116660 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application with Lock( Aug 05 19:49:37.116660 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:49:37.116660 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:49:37.116660 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:49:37.116660 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:49:37.116660 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:49:37.116660 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:49:37.117601 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:49:37.117601 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:49:37.117601 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 05 19:49:37.117601 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 05 19:49:37.117601 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 05 19:49:37.117601 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 05 19:49:37.117601 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 05 19:49:37.117601 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 05 19:49:37.117601 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 05 19:49:37.117601 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Aug 05 19:49:37.117601 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 242/492] 10.222.0.27 () {56 vars in 1093 bytes} [Fri Aug 5 19:49:36 2022] GET /identity/v3/users/nova => generated 82 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:49:37.123265 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be74aae1-8e57-4530-8c96-b81bf2fe78f6 None None] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:37.123265 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be74aae1-8e57-4530-8c96-b81bf2fe78f6 None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:37.123265 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be74aae1-8e57-4530-8c96-b81bf2fe78f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:37.124084 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a2bca5-2cd2-47ae-8e9c-40fca10d49fd None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:37.138486 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.auth.core [None req-be74aae1-8e57-4530-8c96-b81bf2fe78f6 None None] MFA Rules not processed for user `03d1834a64cd4a43829827cb99a72747`. Rule list: `[]` (Enabled: `True`). {{(pid=80364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:49:37.140262 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-be74aae1-8e57-4530-8c96-b81bf2fe78f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:49:37.169807 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327087da-3ab4-4751-9efc-51beb97adc0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:37.171781 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327087da-3ab4-4751-9efc-51beb97adc0a None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:37.172544 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327087da-3ab4-4751-9efc-51beb97adc0a None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:37.173128 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327087da-3ab4-4751-9efc-51beb97adc0a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:37.187228 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-327087da-3ab4-4751-9efc-51beb97adc0a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:37.189212 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-327087da-3ab4-4751-9efc-51beb97adc0a None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:37.209072 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-327087da-3ab4-4751-9efc-51beb97adc0a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:49:37.209072 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:49:37.209072 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:49:37.209072 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:49:37.209072 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:49:37.209072 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:49:37.209072 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:49:37.209072 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:49:37.209072 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:49:37.209072 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:49:37.209072 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:49:37.209072 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:49:37.209072 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 19:49:37.209072 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 19:49:37.209072 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 19:49:37.209971 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 19:49:37.209971 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:49:37.209971 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:49:37.209971 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:49:37.209971 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:49:37.209971 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:49:37.209971 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:49:37.209971 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:49:37.209971 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Aug 05 19:49:37.209971 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:49:37.209971 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:49:37.209971 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:49:37.209971 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:49:37.209971 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:49:37.209971 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:49:37.209971 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:49:37.209971 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:49:37.210805 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 19:49:37.210805 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 19:49:37.210805 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:49:37.210805 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Aug 05 19:49:37.211012 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 243/493] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Aug 5 19:49:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 19:49:37.211012 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be74aae1-8e57-4530-8c96-b81bf2fe78f6 None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:37.213336 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 251/494] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Aug 5 19:49:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 05 19:49:37.221062 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327087da-3ab4-4751-9efc-51beb97adc0a None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:37.296213 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:37.330732 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be74aae1-8e57-4530-8c96-b81bf2fe78f6 None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:37.365609 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327087da-3ab4-4751-9efc-51beb97adc0a None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:37.368338 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:37.374818 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b8affb-f75c-4570-9e73-0f13d06e964f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:37.375707 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b8affb-f75c-4570-9e73-0f13d06e964f None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:37.377074 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b8affb-f75c-4570-9e73-0f13d06e964f None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:37.377553 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b8affb-f75c-4570-9e73-0f13d06e964f None admin] PATH_INFO: `/v3/users` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:37.378631 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b8affb-f75c-4570-9e73-0f13d06e964f None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:49:37.379151 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b8affb-f75c-4570-9e73-0f13d06e964f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:37.380977 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b8affb-f75c-4570-9e73-0f13d06e964f None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:37.404165 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9298d058-bc91-4a10-94d2-e3c13d06fc7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:37.404852 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9298d058-bc91-4a10-94d2-e3c13d06fc7e None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:37.405197 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9298d058-bc91-4a10-94d2-e3c13d06fc7e None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:37.405979 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9298d058-bc91-4a10-94d2-e3c13d06fc7e None admin] PATH_INFO: `/v3/domains` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:37.406742 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9298d058-bc91-4a10-94d2-e3c13d06fc7e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:49:37.407043 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9298d058-bc91-4a10-94d2-e3c13d06fc7e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:37.408464 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9298d058-bc91-4a10-94d2-e3c13d06fc7e None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:37.422295 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 252/495] 10.222.0.27 () {56 vars in 1109 bytes} [Fri Aug 5 19:49:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:49:37.424902 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9298d058-bc91-4a10-94d2-e3c13d06fc7e None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:37.426943 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 244/496] 10.222.0.27 () {56 vars in 1133 bytes} [Fri Aug 5 19:49:37 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:49:37.427833 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b8affb-f75c-4570-9e73-0f13d06e964f None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:37.533782 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9298d058-bc91-4a10-94d2-e3c13d06fc7e None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:37.549518 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b8affb-f75c-4570-9e73-0f13d06e964f None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:37.573259 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e90153a-d280-46d4-9d7f-43a26950a936 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:37.573892 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e90153a-d280-46d4-9d7f-43a26950a936 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:37.574173 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e90153a-d280-46d4-9d7f-43a26950a936 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:37.574454 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e90153a-d280-46d4-9d7f-43a26950a936 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:37.583289 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:37.586801 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e90153a-d280-46d4-9d7f-43a26950a936 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 19:49:37.590237 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe7391a-bd77-400c-8ce7-e54342151963 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:37.590869 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe7391a-bd77-400c-8ce7-e54342151963 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:37.591145 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe7391a-bd77-400c-8ce7-e54342151963 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:37.591418 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe7391a-bd77-400c-8ce7-e54342151963 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:37.593061 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-1e90153a-d280-46d4-9d7f-43a26950a936 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 05 19:49:37.593061 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:49:37.593061 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:49:37.593061 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:49:37.593061 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:49:37.593061 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:49:37.593061 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:49:37.593061 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:49:37.593061 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:49:37.593061 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:49:37.593061 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:49:37.593061 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:49:37.593061 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 19:49:37.593061 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 19:49:37.593061 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 19:49:37.593061 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 19:49:37.594037 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 19:49:37.594037 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 19:49:37.594037 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 19:49:37.594037 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 19:49:37.594037 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:49:37.594037 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:49:37.594037 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:49:37.594037 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:49:37.594037 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:49:37.594037 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:49:37.594037 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:49:37.594037 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Aug 05 19:49:37.594037 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:49:37.594037 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:49:37.594037 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:49:37.594037 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:49:37.594904 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:49:37.594904 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:49:37.594904 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:49:37.594904 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:49:37.594904 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 19:49:37.594904 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 19:49:37.594904 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 19:49:37.594904 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 19:49:37.594904 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 19:49:37.594904 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 19:49:37.594904 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 05 19:49:37.594904 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Aug 05 19:49:37.597226 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 253/497] 10.222.0.27 () {56 vars in 1102 bytes} [Fri Aug 5 19:49:37 2022] GET /identity/v3/roles/service => generated 85 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:49:37.598145 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e90153a-d280-46d4-9d7f-43a26950a936 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:37.605175 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe7391a-bd77-400c-8ce7-e54342151963 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:37.606636 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe7391a-bd77-400c-8ce7-e54342151963 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:37.618801 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application [None req-5fe7391a-bd77-400c-8ce7-e54342151963 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 05 19:49:37.618801 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:49:37.618801 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:49:37.618801 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:49:37.618801 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:49:37.618801 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:49:37.618801 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:49:37.618801 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:49:37.618801 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:49:37.618801 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:49:37.618801 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:49:37.618801 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:49:37.618801 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 05 19:49:37.618801 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 05 19:49:37.618801 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 05 19:49:37.618801 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 05 19:49:37.619701 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:49:37.619701 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:49:37.619701 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 05 19:49:37.619701 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 19:49:37.619701 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 05 19:49:37.619701 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 19:49:37.619701 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:49:37.619701 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:49:37.619701 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:49:37.619701 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:49:37.619701 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:49:37.619701 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application with Lock( Aug 05 19:49:37.619701 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:49:37.619701 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:49:37.619701 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:49:37.619701 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:49:37.619701 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:49:37.619701 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:49:37.620673 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:49:37.620673 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:49:37.620673 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 05 19:49:37.620673 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 05 19:49:37.620673 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 05 19:49:37.620673 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 05 19:49:37.620673 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 05 19:49:37.620673 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 05 19:49:37.620673 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 05 19:49:37.620673 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Aug 05 19:49:37.620673 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 245/498] 10.222.0.27 () {56 vars in 1099 bytes} [Fri Aug 5 19:49:37 2022] GET /identity/v3/users/cinder => generated 84 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:49:37.621877 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0baba4-a8a3-4da4-8a4a-882fd6b31467 None None] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:37.621877 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0baba4-a8a3-4da4-8a4a-882fd6b31467 None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:37.621877 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0baba4-a8a3-4da4-8a4a-882fd6b31467 None None] PATH_INFO: `/` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:37.622905 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 246/499] 10.222.0.27 () {54 vars in 927 bytes} [Fri Aug 5 19:49:37 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:49:37.623405 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0baba4-a8a3-4da4-8a4a-882fd6b31467 None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:37.634329 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:37.710110 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e90153a-d280-46d4-9d7f-43a26950a936 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:37.735580 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0baba4-a8a3-4da4-8a4a-882fd6b31467 None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:37.743863 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86fb15d-32e4-472b-9518-7776b9f5336a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:37.744986 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86fb15d-32e4-472b-9518-7776b9f5336a None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:37.745262 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86fb15d-32e4-472b-9518-7776b9f5336a None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:37.745536 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86fb15d-32e4-472b-9518-7776b9f5336a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:37.770192 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f924cf-1531-492d-a3b6-f7eb7af803d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:37.770804 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f924cf-1531-492d-a3b6-f7eb7af803d1 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:37.771089 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f924cf-1531-492d-a3b6-f7eb7af803d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:37.771371 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f924cf-1531-492d-a3b6-f7eb7af803d1 None admin] PATH_INFO: `/v3/roles` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:37.772083 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f924cf-1531-492d-a3b6-f7eb7af803d1 None admin] RBAC: Adding query filter params (name=service) {{(pid=80364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:49:37.772386 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f924cf-1531-492d-a3b6-f7eb7af803d1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:37.773802 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f924cf-1531-492d-a3b6-f7eb7af803d1 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:37.783425 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 247/500] 10.222.0.27 () {56 vars in 1103 bytes} [Fri Aug 5 19:49:37 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:49:37.784207 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f924cf-1531-492d-a3b6-f7eb7af803d1 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:37.822023 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86fb15d-32e4-472b-9518-7776b9f5336a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:37.823536 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86fb15d-32e4-472b-9518-7776b9f5336a None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:37.835558 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:37.847269 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:37.884060 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f924cf-1531-492d-a3b6-f7eb7af803d1 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:37.917107 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b02c049-c3e0-47e5-9512-375b126c6533 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:37.917720 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b02c049-c3e0-47e5-9512-375b126c6533 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:37.918002 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b02c049-c3e0-47e5-9512-375b126c6533 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:37.918273 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b02c049-c3e0-47e5-9512-375b126c6533 None admin] PATH_INFO: `/v3/users` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:37.919054 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b02c049-c3e0-47e5-9512-375b126c6533 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:49:37.919349 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b02c049-c3e0-47e5-9512-375b126c6533 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:37.921366 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b02c049-c3e0-47e5-9512-375b126c6533 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:37.928102 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 254/501] 10.222.0.27 () {58 vars in 1282 bytes} [Fri Aug 5 19:49:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 05 19:49:37.929285 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618e555c-e20a-4b59-b49d-4ca8553f3411 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:37.929581 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618e555c-e20a-4b59-b49d-4ca8553f3411 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:37.929860 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618e555c-e20a-4b59-b49d-4ca8553f3411 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:37.942406 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 248/502] 10.222.0.27 () {56 vars in 1137 bytes} [Fri Aug 5 19:49:37 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:49:37.943623 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ddf361-65c3-4f58-9bbd-1cba1edde41f None None] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:37.943907 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ddf361-65c3-4f58-9bbd-1cba1edde41f None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:37.944237 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ddf361-65c3-4f58-9bbd-1cba1edde41f None None] PATH_INFO: `/` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:37.945608 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 249/503] 10.222.0.27 () {54 vars in 927 bytes} [Fri Aug 5 19:49:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:49:37.947305 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ddf361-65c3-4f58-9bbd-1cba1edde41f None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:37.948102 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-618e555c-e20a-4b59-b49d-4ca8553f3411 None None] MFA Rules not processed for user `03d1834a64cd4a43829827cb99a72747`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:49:37.949359 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-618e555c-e20a-4b59-b49d-4ca8553f3411 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:49:38.015610 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 255/504] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Aug 5 19:49:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 19:49:38.016453 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618e555c-e20a-4b59-b49d-4ca8553f3411 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:38.052634 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ddf361-65c3-4f58-9bbd-1cba1edde41f None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:38.063876 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:38.072746 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:38.087690 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf41a30c-b7e7-4f59-af2b-698fbc6f5e18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:38.088337 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf41a30c-b7e7-4f59-af2b-698fbc6f5e18 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:38.088657 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf41a30c-b7e7-4f59-af2b-698fbc6f5e18 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:38.088928 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf41a30c-b7e7-4f59-af2b-698fbc6f5e18 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:38.099740 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf41a30c-b7e7-4f59-af2b-698fbc6f5e18 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:38.101241 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf41a30c-b7e7-4f59-af2b-698fbc6f5e18 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:38.117071 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application [None req-bf41a30c-b7e7-4f59-af2b-698fbc6f5e18 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:49:38.117071 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:49:38.117071 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:49:38.117071 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:49:38.117071 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:49:38.117071 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:49:38.117071 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:49:38.117071 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:49:38.117071 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:49:38.117071 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:49:38.117071 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:49:38.117071 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:49:38.117071 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 19:49:38.117071 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 19:49:38.117071 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 19:49:38.117979 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 19:49:38.117979 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:49:38.117979 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:49:38.117979 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:49:38.117979 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:49:38.117979 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:49:38.117979 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:49:38.117979 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:49:38.117979 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application with Lock( Aug 05 19:49:38.117979 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:49:38.117979 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:49:38.117979 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:49:38.117979 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:49:38.117979 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:49:38.117979 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:49:38.117979 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:49:38.117979 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:49:38.118837 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 19:49:38.118837 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 19:49:38.118837 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:49:38.118837 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Aug 05 19:49:38.118837 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 250/505] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Aug 5 19:49:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:49:38.119102 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf41a30c-b7e7-4f59-af2b-698fbc6f5e18 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:38.127106 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618e555c-e20a-4b59-b49d-4ca8553f3411 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:38.161274 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a114bd3-ee03-4f6d-ac83-f5a962a0eed9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:38.161824 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a114bd3-ee03-4f6d-ac83-f5a962a0eed9 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:38.162100 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a114bd3-ee03-4f6d-ac83-f5a962a0eed9 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:38.162364 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a114bd3-ee03-4f6d-ac83-f5a962a0eed9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:38.173040 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a114bd3-ee03-4f6d-ac83-f5a962a0eed9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:38.174510 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a114bd3-ee03-4f6d-ac83-f5a962a0eed9 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:38.190382 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-2a114bd3-ee03-4f6d-ac83-f5a962a0eed9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:49:38.190382 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:49:38.190382 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:49:38.190382 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:49:38.190382 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:49:38.190382 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:49:38.190382 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:49:38.190382 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:49:38.190382 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:49:38.190382 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:49:38.190382 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:49:38.190382 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:49:38.190382 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 19:49:38.190382 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 19:49:38.190382 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 19:49:38.191260 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 19:49:38.191260 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:49:38.191260 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:49:38.191260 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:49:38.191260 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:49:38.191260 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:49:38.191260 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:49:38.191260 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:49:38.191260 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Aug 05 19:49:38.191260 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:49:38.191260 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:49:38.191260 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:49:38.191260 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:49:38.191260 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:49:38.191260 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:49:38.191260 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:49:38.191260 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:49:38.192110 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 19:49:38.192110 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 19:49:38.192110 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:49:38.192110 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Aug 05 19:49:38.192110 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 256/506] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Aug 5 19:49:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:49:38.192836 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224f62c9-22f5-4981-8358-8901cd118eb8 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:38.193052 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224f62c9-22f5-4981-8358-8901cd118eb8 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:38.193326 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224f62c9-22f5-4981-8358-8901cd118eb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:38.210962 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-224f62c9-22f5-4981-8358-8901cd118eb8 None None] MFA Rules not processed for user `03d1834a64cd4a43829827cb99a72747`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:49:38.212144 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-224f62c9-22f5-4981-8358-8901cd118eb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:49:38.225203 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf41a30c-b7e7-4f59-af2b-698fbc6f5e18 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:38.257903 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee8297b-7cff-4cc1-a48e-8c4edf8b6f9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:38.258487 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee8297b-7cff-4cc1-a48e-8c4edf8b6f9d None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:38.258766 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee8297b-7cff-4cc1-a48e-8c4edf8b6f9d None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:38.260081 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee8297b-7cff-4cc1-a48e-8c4edf8b6f9d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:38.292521 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 257/507] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Aug 5 19:49:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 19:49:38.292521 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224f62c9-22f5-4981-8358-8901cd118eb8 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:38.292770 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:38.316521 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:38.327311 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee8297b-7cff-4cc1-a48e-8c4edf8b6f9d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:38.328825 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee8297b-7cff-4cc1-a48e-8c4edf8b6f9d None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:38.385091 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224f62c9-22f5-4981-8358-8901cd118eb8 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:38.419824 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78078dd1-1a0f-46f7-b09b-f06798411722 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:38.420391 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78078dd1-1a0f-46f7-b09b-f06798411722 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:38.420731 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78078dd1-1a0f-46f7-b09b-f06798411722 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:38.421508 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78078dd1-1a0f-46f7-b09b-f06798411722 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:38.428271 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 251/508] 10.222.0.27 () {58 vars in 1282 bytes} [Fri Aug 5 19:49:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 05 19:49:38.428963 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee8297b-7cff-4cc1-a48e-8c4edf8b6f9d None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:38.435785 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78078dd1-1a0f-46f7-b09b-f06798411722 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 19:49:38.440703 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-78078dd1-1a0f-46f7-b09b-f06798411722 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 05 19:49:38.440703 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:49:38.440703 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:49:38.440703 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:49:38.440703 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:49:38.440703 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:49:38.440703 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:49:38.440703 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:49:38.440703 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:49:38.440703 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:49:38.440703 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:49:38.440703 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:49:38.440703 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 19:49:38.440703 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 19:49:38.440703 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 19:49:38.440703 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 19:49:38.441763 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 19:49:38.441763 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 19:49:38.441763 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 19:49:38.441763 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 19:49:38.441763 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:49:38.441763 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:49:38.441763 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:49:38.441763 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:49:38.441763 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:49:38.441763 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:49:38.441763 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:49:38.441763 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Aug 05 19:49:38.441763 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:49:38.441763 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:49:38.441763 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:49:38.441763 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:49:38.442577 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:49:38.442577 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:49:38.442577 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:49:38.442577 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:49:38.442577 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 19:49:38.442577 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 19:49:38.442577 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 19:49:38.442577 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 19:49:38.442577 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 19:49:38.442577 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 19:49:38.442577 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 05 19:49:38.442577 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Aug 05 19:49:38.442577 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 258/509] 10.222.0.27 () {56 vars in 1102 bytes} [Fri Aug 5 19:49:38 2022] GET /identity/v3/roles/service => generated 85 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:49:38.443229 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78078dd1-1a0f-46f7-b09b-f06798411722 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:38.494342 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:38.533584 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee8297b-7cff-4cc1-a48e-8c4edf8b6f9d None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:38.537626 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:38.543952 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78078dd1-1a0f-46f7-b09b-f06798411722 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:38.566328 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc110af-c28c-4d17-ad01-aa8e86b3dc22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:38.566937 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc110af-c28c-4d17-ad01-aa8e86b3dc22 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:38.567217 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc110af-c28c-4d17-ad01-aa8e86b3dc22 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:38.567487 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc110af-c28c-4d17-ad01-aa8e86b3dc22 None admin] PATH_INFO: `/v3/domains` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:38.568173 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc110af-c28c-4d17-ad01-aa8e86b3dc22 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:49:38.568475 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc110af-c28c-4d17-ad01-aa8e86b3dc22 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:38.569865 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc110af-c28c-4d17-ad01-aa8e86b3dc22 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:38.577442 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785bdec8-6fb9-4ca4-8fe0-7b1d01aa6ec7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:38.578131 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785bdec8-6fb9-4ca4-8fe0-7b1d01aa6ec7 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:38.578378 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785bdec8-6fb9-4ca4-8fe0-7b1d01aa6ec7 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:38.578639 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785bdec8-6fb9-4ca4-8fe0-7b1d01aa6ec7 None admin] PATH_INFO: `/v3/domains` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:38.579339 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785bdec8-6fb9-4ca4-8fe0-7b1d01aa6ec7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:49:38.579643 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785bdec8-6fb9-4ca4-8fe0-7b1d01aa6ec7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:38.581037 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785bdec8-6fb9-4ca4-8fe0-7b1d01aa6ec7 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:38.581612 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 252/510] 10.222.0.27 () {56 vars in 1109 bytes} [Fri Aug 5 19:49:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:49:38.582564 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc110af-c28c-4d17-ad01-aa8e86b3dc22 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:38.592918 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 259/511] 10.222.0.27 () {56 vars in 1109 bytes} [Fri Aug 5 19:49:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:49:38.593740 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785bdec8-6fb9-4ca4-8fe0-7b1d01aa6ec7 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:38.682814 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc110af-c28c-4d17-ad01-aa8e86b3dc22 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:38.694904 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785bdec8-6fb9-4ca4-8fe0-7b1d01aa6ec7 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:38.704330 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:38.715867 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a64491-1892-4da3-859f-c949baf29725 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:38.716518 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a64491-1892-4da3-859f-c949baf29725 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:38.716799 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a64491-1892-4da3-859f-c949baf29725 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:38.717073 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a64491-1892-4da3-859f-c949baf29725 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:38.727682 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a64491-1892-4da3-859f-c949baf29725 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:38.728467 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc731ce-f0ac-4ed0-a5c3-03ec2d273751 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:38.729138 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc731ce-f0ac-4ed0-a5c3-03ec2d273751 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:38.729260 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a64491-1892-4da3-859f-c949baf29725 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:38.729419 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc731ce-f0ac-4ed0-a5c3-03ec2d273751 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:38.729692 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc731ce-f0ac-4ed0-a5c3-03ec2d273751 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:38.740935 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc731ce-f0ac-4ed0-a5c3-03ec2d273751 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:38.742361 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc731ce-f0ac-4ed0-a5c3-03ec2d273751 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:38.742761 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application [None req-e1a64491-1892-4da3-859f-c949baf29725 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:49:38.742761 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:49:38.742761 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:49:38.742761 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:49:38.742761 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:49:38.742761 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:49:38.742761 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:49:38.742761 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:49:38.742761 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:49:38.742761 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:49:38.742761 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:49:38.742761 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:49:38.742761 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 19:49:38.742761 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 19:49:38.742761 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 19:49:38.743650 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 19:49:38.743650 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:49:38.743650 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:49:38.743650 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:49:38.743650 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:49:38.743650 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:49:38.743650 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:49:38.743650 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:49:38.743650 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application with Lock( Aug 05 19:49:38.743650 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:49:38.743650 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:49:38.743650 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:49:38.743650 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:49:38.743650 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:49:38.743650 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:49:38.743650 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:49:38.743650 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:49:38.744702 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 19:49:38.744702 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 19:49:38.744702 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:49:38.744702 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Aug 05 19:49:38.744702 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 253/512] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Aug 5 19:49:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:49:38.745041 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a64491-1892-4da3-859f-c949baf29725 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:38.758295 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-8dc731ce-f0ac-4ed0-a5c3-03ec2d273751 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:49:38.758295 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:49:38.758295 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:49:38.758295 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:49:38.758295 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:49:38.758295 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:49:38.758295 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:49:38.758295 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:49:38.758295 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:49:38.758295 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:49:38.758295 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:49:38.758295 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:49:38.758295 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 19:49:38.758295 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 19:49:38.758295 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 19:49:38.759300 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 19:49:38.759300 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:49:38.759300 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:49:38.759300 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:49:38.759300 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:49:38.759300 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:49:38.759300 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:49:38.759300 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:49:38.759300 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Aug 05 19:49:38.759300 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:49:38.759300 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:49:38.759300 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:49:38.759300 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:49:38.759300 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:49:38.759300 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:49:38.759300 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:49:38.759300 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:49:38.760332 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 19:49:38.760332 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 19:49:38.760332 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:49:38.760332 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Aug 05 19:49:38.763928 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 260/513] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Aug 5 19:49:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:49:38.765139 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc731ce-f0ac-4ed0-a5c3-03ec2d273751 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:38.775922 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:38.844885 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a64491-1892-4da3-859f-c949baf29725 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:38.863723 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc731ce-f0ac-4ed0-a5c3-03ec2d273751 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:38.877649 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee9baa9-9f61-4d34-9ed6-69c3eebef6ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:38.878254 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee9baa9-9f61-4d34-9ed6-69c3eebef6ab None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:38.878532 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee9baa9-9f61-4d34-9ed6-69c3eebef6ab None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:38.878802 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee9baa9-9f61-4d34-9ed6-69c3eebef6ab None admin] PATH_INFO: `/v3/roles` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:38.879507 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee9baa9-9f61-4d34-9ed6-69c3eebef6ab None admin] RBAC: Adding query filter params (name=service) {{(pid=80364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:49:38.879808 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee9baa9-9f61-4d34-9ed6-69c3eebef6ab None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:38.881192 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee9baa9-9f61-4d34-9ed6-69c3eebef6ab None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:38.889121 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 254/514] 10.222.0.27 () {56 vars in 1103 bytes} [Fri Aug 5 19:49:38 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:49:38.890388 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee9baa9-9f61-4d34-9ed6-69c3eebef6ab None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:38.898575 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2a5bea-f21e-4239-bf1c-2897a6693525 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:38.899442 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2a5bea-f21e-4239-bf1c-2897a6693525 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:38.900062 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2a5bea-f21e-4239-bf1c-2897a6693525 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:38.900337 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2a5bea-f21e-4239-bf1c-2897a6693525 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:38.907873 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d2a5bea-f21e-4239-bf1c-2897a6693525 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:38.909329 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d2a5bea-f21e-4239-bf1c-2897a6693525 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:38.921566 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-4d2a5bea-f21e-4239-bf1c-2897a6693525 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 05 19:49:38.921566 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:49:38.921566 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:49:38.921566 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:49:38.921566 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:49:38.921566 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:49:38.921566 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:49:38.921566 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:49:38.921566 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:49:38.921566 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:49:38.921566 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:49:38.921566 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:49:38.921566 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 05 19:49:38.921566 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 05 19:49:38.921566 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 05 19:49:38.921566 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 05 19:49:38.922603 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:49:38.922603 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:49:38.922603 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 05 19:49:38.922603 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 19:49:38.922603 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 05 19:49:38.922603 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 19:49:38.922603 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:49:38.922603 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:49:38.922603 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:49:38.922603 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:49:38.922603 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:49:38.922603 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Aug 05 19:49:38.922603 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:49:38.922603 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:49:38.922603 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:49:38.922603 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:49:38.922603 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:49:38.922603 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:49:38.923512 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:49:38.923512 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:49:38.923512 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 05 19:49:38.923512 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 05 19:49:38.923512 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 05 19:49:38.923512 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 05 19:49:38.923512 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 05 19:49:38.923512 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 05 19:49:38.923512 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 05 19:49:38.923512 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Aug 05 19:49:38.923512 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 261/515] 10.222.0.27 () {56 vars in 1099 bytes} [Fri Aug 5 19:49:38 2022] GET /identity/v3/users/glance => generated 84 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:49:38.923512 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2a5bea-f21e-4239-bf1c-2897a6693525 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:38.925230 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:38.990050 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee9baa9-9f61-4d34-9ed6-69c3eebef6ab None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:38.991876 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:39.023855 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99ac74e-4098-4c9b-b110-099c5a702fd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:39.024574 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99ac74e-4098-4c9b-b110-099c5a702fd9 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:39.026441 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99ac74e-4098-4c9b-b110-099c5a702fd9 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:39.026441 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99ac74e-4098-4c9b-b110-099c5a702fd9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:39.027602 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2a5bea-f21e-4239-bf1c-2897a6693525 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:39.032840 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a99ac74e-4098-4c9b-b110-099c5a702fd9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:39.034285 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a99ac74e-4098-4c9b-b110-099c5a702fd9 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:39.046035 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application [None req-a99ac74e-4098-4c9b-b110-099c5a702fd9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 19:49:39.046035 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:49:39.046035 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:49:39.046035 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:49:39.046035 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:49:39.046035 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:49:39.046035 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:49:39.046035 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:49:39.046035 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:49:39.046035 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:49:39.046035 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:49:39.046035 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:49:39.046035 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 05 19:49:39.046035 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 05 19:49:39.046035 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 05 19:49:39.047013 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 05 19:49:39.047013 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:49:39.047013 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:49:39.047013 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:49:39.047013 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:49:39.047013 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:49:39.047013 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:49:39.047013 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:49:39.047013 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application with Lock( Aug 05 19:49:39.047013 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:49:39.047013 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:49:39.047013 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:49:39.047013 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:49:39.047013 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:49:39.047013 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:49:39.047013 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:49:39.047013 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:49:39.047869 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 05 19:49:39.047869 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 05 19:49:39.047869 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 05 19:49:39.047869 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 05 19:49:39.047869 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 05 19:49:39.047869 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 05 19:49:39.047869 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 19:49:39.047869 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Aug 05 19:49:39.047869 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 255/516] 10.222.0.27 () {56 vars in 1111 bytes} [Fri Aug 5 19:49:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:49:39.048330 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99ac74e-4098-4c9b-b110-099c5a702fd9 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:39.065836 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede2bdb4-523e-40d4-9c4f-6cfaf283a30c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:39.066451 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede2bdb4-523e-40d4-9c4f-6cfaf283a30c None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:39.066734 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede2bdb4-523e-40d4-9c4f-6cfaf283a30c None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:39.067006 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede2bdb4-523e-40d4-9c4f-6cfaf283a30c None admin] PATH_INFO: `/v3/domains` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:39.067701 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede2bdb4-523e-40d4-9c4f-6cfaf283a30c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:49:39.068004 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede2bdb4-523e-40d4-9c4f-6cfaf283a30c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:39.069419 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede2bdb4-523e-40d4-9c4f-6cfaf283a30c None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:39.081117 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 262/517] 10.222.0.27 () {56 vars in 1109 bytes} [Fri Aug 5 19:49:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:49:39.082506 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede2bdb4-523e-40d4-9c4f-6cfaf283a30c None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:39.142621 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:39.148470 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99ac74e-4098-4c9b-b110-099c5a702fd9 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:39.181914 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e04cdc-04ea-47fb-9eaa-3349b0da3717 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:39.182516 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e04cdc-04ea-47fb-9eaa-3349b0da3717 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:39.182734 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede2bdb4-523e-40d4-9c4f-6cfaf283a30c None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:39.182827 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e04cdc-04ea-47fb-9eaa-3349b0da3717 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:39.183059 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e04cdc-04ea-47fb-9eaa-3349b0da3717 None admin] PATH_INFO: `/v3/domains` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:39.183744 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e04cdc-04ea-47fb-9eaa-3349b0da3717 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:49:39.184055 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e04cdc-04ea-47fb-9eaa-3349b0da3717 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:39.185488 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e04cdc-04ea-47fb-9eaa-3349b0da3717 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:39.197659 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 256/518] 10.222.0.27 () {56 vars in 1109 bytes} [Fri Aug 5 19:49:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:49:39.198449 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e04cdc-04ea-47fb-9eaa-3349b0da3717 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:39.209620 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:39.218761 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c189ff-eb87-4a36-b9d6-73f2f0277a16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:39.219397 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c189ff-eb87-4a36-b9d6-73f2f0277a16 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:39.219683 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c189ff-eb87-4a36-b9d6-73f2f0277a16 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:39.219958 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c189ff-eb87-4a36-b9d6-73f2f0277a16 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:39.230014 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c189ff-eb87-4a36-b9d6-73f2f0277a16 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:39.231452 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c189ff-eb87-4a36-b9d6-73f2f0277a16 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:39.245812 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-f2c189ff-eb87-4a36-b9d6-73f2f0277a16 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:49:39.245812 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:49:39.245812 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:49:39.245812 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:49:39.245812 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:49:39.245812 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:49:39.245812 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:49:39.245812 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:49:39.245812 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:49:39.245812 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:49:39.245812 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:49:39.245812 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:49:39.245812 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 19:49:39.245812 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 19:49:39.245812 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 19:49:39.246638 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 19:49:39.246638 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:49:39.246638 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:49:39.246638 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:49:39.246638 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:49:39.246638 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:49:39.246638 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:49:39.246638 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:49:39.246638 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Aug 05 19:49:39.246638 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:49:39.246638 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:49:39.246638 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:49:39.246638 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:49:39.246638 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:49:39.246638 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:49:39.246638 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:49:39.246638 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:49:39.247469 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 19:49:39.247469 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 19:49:39.247469 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:49:39.247469 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Aug 05 19:49:39.247469 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 263/519] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Aug 5 19:49:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:49:39.247741 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c189ff-eb87-4a36-b9d6-73f2f0277a16 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:39.306470 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e04cdc-04ea-47fb-9eaa-3349b0da3717 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:39.340667 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece22381-64aa-43c4-888f-1b99ce2cc61d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:39.341385 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece22381-64aa-43c4-888f-1b99ce2cc61d None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:39.341666 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece22381-64aa-43c4-888f-1b99ce2cc61d None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:39.341939 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece22381-64aa-43c4-888f-1b99ce2cc61d None admin] PATH_INFO: `/v3/users` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:39.342721 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece22381-64aa-43c4-888f-1b99ce2cc61d None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:49:39.343023 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece22381-64aa-43c4-888f-1b99ce2cc61d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:39.344465 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece22381-64aa-43c4-888f-1b99ce2cc61d None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:39.350707 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c189ff-eb87-4a36-b9d6-73f2f0277a16 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:39.363815 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 257/520] 10.222.0.27 () {56 vars in 1137 bytes} [Fri Aug 5 19:49:39 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:49:39.365080 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece22381-64aa-43c4-888f-1b99ce2cc61d None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:39.380585 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:39.388934 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8755fec2-b327-4545-bcc0-565814292874 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:39.390060 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8755fec2-b327-4545-bcc0-565814292874 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:39.390342 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8755fec2-b327-4545-bcc0-565814292874 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:39.390616 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8755fec2-b327-4545-bcc0-565814292874 None admin] PATH_INFO: `/v3/projects` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:39.391371 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8755fec2-b327-4545-bcc0-565814292874 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:49:39.391677 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8755fec2-b327-4545-bcc0-565814292874 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:39.393186 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8755fec2-b327-4545-bcc0-565814292874 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:39.411955 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 264/521] 10.222.0.27 () {56 vars in 1148 bytes} [Fri Aug 5 19:49:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:49:39.412797 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8755fec2-b327-4545-bcc0-565814292874 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:39.457430 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:39.468954 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece22381-64aa-43c4-888f-1b99ce2cc61d None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:39.502158 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85537f0f-92a5-4d0a-857d-30f5c10b3bad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:39.502696 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85537f0f-92a5-4d0a-857d-30f5c10b3bad None admin] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:39.502974 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85537f0f-92a5-4d0a-857d-30f5c10b3bad None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:39.503244 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85537f0f-92a5-4d0a-857d-30f5c10b3bad None admin] PATH_INFO: `/v3/users` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:39.503918 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85537f0f-92a5-4d0a-857d-30f5c10b3bad None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:39.505380 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85537f0f-92a5-4d0a-857d-30f5c10b3bad None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:39.515490 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8755fec2-b327-4545-bcc0-565814292874 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:39.525799 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 258/522] 10.222.0.27 () {60 vars in 1132 bytes} [Fri Aug 5 19:49:39 2022] POST /identity/v3/users => generated 247 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 19:49:39.526609 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85537f0f-92a5-4d0a-857d-30f5c10b3bad None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:39.554557 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318ef24c-2c8f-4cec-9211-fa9fda6763a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:39.555562 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318ef24c-2c8f-4cec-9211-fa9fda6763a3 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:39.556011 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318ef24c-2c8f-4cec-9211-fa9fda6763a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:39.556470 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318ef24c-2c8f-4cec-9211-fa9fda6763a3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:39.567177 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-318ef24c-2c8f-4cec-9211-fa9fda6763a3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:39.569678 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-318ef24c-2c8f-4cec-9211-fa9fda6763a3 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:39.586748 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-318ef24c-2c8f-4cec-9211-fa9fda6763a3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 19:49:39.586748 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:49:39.586748 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:49:39.586748 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:49:39.586748 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:49:39.586748 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:49:39.586748 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:49:39.586748 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:49:39.586748 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:49:39.586748 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:49:39.586748 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:49:39.586748 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:49:39.586748 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 05 19:49:39.586748 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 05 19:49:39.586748 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 05 19:49:39.587986 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 05 19:49:39.587986 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:49:39.587986 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:49:39.587986 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:49:39.587986 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:49:39.587986 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:49:39.587986 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:49:39.587986 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:49:39.587986 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Aug 05 19:49:39.587986 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:49:39.587986 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:49:39.587986 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:49:39.587986 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:49:39.587986 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:49:39.587986 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:49:39.587986 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:49:39.587986 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:49:39.589053 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 05 19:49:39.589053 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 05 19:49:39.589053 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 05 19:49:39.589053 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 05 19:49:39.589053 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 05 19:49:39.589053 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 05 19:49:39.589053 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 19:49:39.589053 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Aug 05 19:49:39.590035 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 265/523] 10.222.0.27 () {56 vars in 1111 bytes} [Fri Aug 5 19:49:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:49:39.590707 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318ef24c-2c8f-4cec-9211-fa9fda6763a3 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:39.617229 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:39.660024 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85537f0f-92a5-4d0a-857d-30f5c10b3bad None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:39.701634 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2e3bdc-9097-49ce-8ad8-63ab9945394a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:39.702408 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2e3bdc-9097-49ce-8ad8-63ab9945394a None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:39.702561 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2e3bdc-9097-49ce-8ad8-63ab9945394a None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:39.703018 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2e3bdc-9097-49ce-8ad8-63ab9945394a None admin] PATH_INFO: `/v3/domains` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:39.703503 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af2e3bdc-9097-49ce-8ad8-63ab9945394a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:49:39.703814 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af2e3bdc-9097-49ce-8ad8-63ab9945394a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:39.705215 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af2e3bdc-9097-49ce-8ad8-63ab9945394a None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:39.706764 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:39.710235 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318ef24c-2c8f-4cec-9211-fa9fda6763a3 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:39.723407 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 259/524] 10.222.0.27 () {56 vars in 1109 bytes} [Fri Aug 5 19:49:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:49:39.724670 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2e3bdc-9097-49ce-8ad8-63ab9945394a None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:39.750535 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08410a37-e87d-46a3-9ee9-778b15060243 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:39.751153 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08410a37-e87d-46a3-9ee9-778b15060243 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:39.751430 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08410a37-e87d-46a3-9ee9-778b15060243 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:39.751699 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08410a37-e87d-46a3-9ee9-778b15060243 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:39.825316 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08410a37-e87d-46a3-9ee9-778b15060243 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:39.826702 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08410a37-e87d-46a3-9ee9-778b15060243 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:39.837089 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2e3bdc-9097-49ce-8ad8-63ab9945394a None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:39.870393 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:39.877683 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d10d3ad-f3a5-4ccd-9eec-234b7a7fb9ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:39.878458 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d10d3ad-f3a5-4ccd-9eec-234b7a7fb9ee None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:39.878837 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d10d3ad-f3a5-4ccd-9eec-234b7a7fb9ee None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:39.879203 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d10d3ad-f3a5-4ccd-9eec-234b7a7fb9ee None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:39.880072 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d10d3ad-f3a5-4ccd-9eec-234b7a7fb9ee None admin] RBAC: Adding query filter params (role.id=6e2bc1ba438d4eb69923ef4659c181f3, scope.project.id=a70a82dcfc574fad90b3f5f00abbba58, user.id=11001a2ad96f4ec8a35aab2c64376875) {{(pid=80364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:49:39.880461 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d10d3ad-f3a5-4ccd-9eec-234b7a7fb9ee None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:39.882005 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d10d3ad-f3a5-4ccd-9eec-234b7a7fb9ee None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:39.887782 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 260/525] 10.222.0.27 () {56 vars in 1374 bytes} [Fri Aug 5 19:49:39 2022] GET /identity/v3/role_assignments?user.id=11001a2ad96f4ec8a35aab2c64376875&scope.project.id=a70a82dcfc574fad90b3f5f00abbba58&role.id=6e2bc1ba438d4eb69923ef4659c181f3 => generated 259 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:49:39.888786 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d10d3ad-f3a5-4ccd-9eec-234b7a7fb9ee None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:39.956899 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:39.962753 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 266/526] 10.222.0.27 () {58 vars in 1282 bytes} [Fri Aug 5 19:49:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 05 19:49:39.963613 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08410a37-e87d-46a3-9ee9-778b15060243 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:40.023604 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d10d3ad-f3a5-4ccd-9eec-234b7a7fb9ee None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:40.065796 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5f082d-3b5c-4dc2-a8da-30907ae71dcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:40.066746 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5f082d-3b5c-4dc2-a8da-30907ae71dcf None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:40.067287 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5f082d-3b5c-4dc2-a8da-30907ae71dcf None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:40.067709 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5f082d-3b5c-4dc2-a8da-30907ae71dcf None admin] PATH_INFO: `/v3/projects` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:40.068660 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc5f082d-3b5c-4dc2-a8da-30907ae71dcf None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:49:40.069126 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc5f082d-3b5c-4dc2-a8da-30907ae71dcf None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:40.071089 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc5f082d-3b5c-4dc2-a8da-30907ae71dcf None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:40.086755 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08410a37-e87d-46a3-9ee9-778b15060243 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:40.096979 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 261/527] 10.222.0.27 () {56 vars in 1148 bytes} [Fri Aug 5 19:49:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:49:40.098124 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5f082d-3b5c-4dc2-a8da-30907ae71dcf None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:40.123877 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7870e3a8-df36-4e0b-a924-6a2b29d65de1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:40.124555 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7870e3a8-df36-4e0b-a924-6a2b29d65de1 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:40.124842 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7870e3a8-df36-4e0b-a924-6a2b29d65de1 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:40.125123 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7870e3a8-df36-4e0b-a924-6a2b29d65de1 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:40.133070 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7870e3a8-df36-4e0b-a924-6a2b29d65de1 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:40.134749 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7870e3a8-df36-4e0b-a924-6a2b29d65de1 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:40.147875 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-7870e3a8-df36-4e0b-a924-6a2b29d65de1 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 05 19:49:40.147875 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:49:40.147875 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:49:40.147875 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:49:40.147875 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:49:40.147875 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:49:40.147875 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:49:40.147875 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:49:40.147875 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:49:40.147875 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:49:40.147875 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:49:40.147875 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:49:40.147875 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 05 19:49:40.147875 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 05 19:49:40.147875 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 05 19:49:40.147875 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 05 19:49:40.149042 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:49:40.149042 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:49:40.149042 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 05 19:49:40.149042 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 19:49:40.149042 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 05 19:49:40.149042 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 19:49:40.149042 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:49:40.149042 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:49:40.149042 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:49:40.149042 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:49:40.149042 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:49:40.149042 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Aug 05 19:49:40.149042 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:49:40.149042 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:49:40.149042 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:49:40.149042 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:49:40.149042 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:49:40.149042 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:49:40.150459 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:49:40.150459 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:49:40.150459 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 05 19:49:40.150459 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 05 19:49:40.150459 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 05 19:49:40.150459 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 05 19:49:40.150459 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 05 19:49:40.150459 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 05 19:49:40.150459 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 05 19:49:40.150459 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Aug 05 19:49:40.154048 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 267/528] 10.222.0.27 () {56 vars in 1102 bytes} [Fri Aug 5 19:49:39 2022] GET /identity/v3/users/neutron => generated 85 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:49:40.155092 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7870e3a8-df36-4e0b-a924-6a2b29d65de1 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:40.165059 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:40.219742 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5f082d-3b5c-4dc2-a8da-30907ae71dcf None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:40.223526 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:40.261186 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e78760-7c86-4724-8b10-ba40a468cbb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:40.262377 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e78760-7c86-4724-8b10-ba40a468cbb0 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:40.262809 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e78760-7c86-4724-8b10-ba40a468cbb0 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:40.263261 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e78760-7c86-4724-8b10-ba40a468cbb0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:40.269190 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7870e3a8-df36-4e0b-a924-6a2b29d65de1 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:40.276180 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e78760-7c86-4724-8b10-ba40a468cbb0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:40.278324 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e78760-7c86-4724-8b10-ba40a468cbb0 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:40.296197 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application [None req-d1e78760-7c86-4724-8b10-ba40a468cbb0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:49:40.296197 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:49:40.296197 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:49:40.296197 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:49:40.296197 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:49:40.296197 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:49:40.296197 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:49:40.296197 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:49:40.296197 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:49:40.296197 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:49:40.296197 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:49:40.296197 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:49:40.296197 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 19:49:40.296197 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 19:49:40.296197 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 19:49:40.297335 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 19:49:40.297335 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:49:40.297335 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:49:40.297335 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:49:40.297335 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:49:40.297335 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:49:40.297335 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:49:40.297335 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:49:40.297335 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application with Lock( Aug 05 19:49:40.297335 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:49:40.297335 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:49:40.297335 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:49:40.297335 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:49:40.297335 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:49:40.297335 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:49:40.297335 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:49:40.297335 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:49:40.298366 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 19:49:40.298366 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 19:49:40.298366 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:49:40.298366 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Aug 05 19:49:40.300432 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 262/529] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Aug 5 19:49:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:49:40.301376 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e78760-7c86-4724-8b10-ba40a468cbb0 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:40.306081 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e8c1da-3a3e-455f-9d3b-fc2e6f206c4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:40.306552 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e8c1da-3a3e-455f-9d3b-fc2e6f206c4a None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:40.306831 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e8c1da-3a3e-455f-9d3b-fc2e6f206c4a None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:40.307107 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e8c1da-3a3e-455f-9d3b-fc2e6f206c4a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:40.307910 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e8c1da-3a3e-455f-9d3b-fc2e6f206c4a None admin] RBAC: Adding query filter params (role.id=6e2bc1ba438d4eb69923ef4659c181f3, scope.project.id=a70a82dcfc574fad90b3f5f00abbba58, user.id=c092a23e14e24265973ff5e899670fc8) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:49:40.308222 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e8c1da-3a3e-455f-9d3b-fc2e6f206c4a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:40.309712 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e8c1da-3a3e-455f-9d3b-fc2e6f206c4a None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:40.322508 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 268/530] 10.222.0.27 () {56 vars in 1374 bytes} [Fri Aug 5 19:49:40 2022] GET /identity/v3/role_assignments?user.id=c092a23e14e24265973ff5e899670fc8&scope.project.id=a70a82dcfc574fad90b3f5f00abbba58&role.id=6e2bc1ba438d4eb69923ef4659c181f3 => generated 259 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:49:40.322508 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e8c1da-3a3e-455f-9d3b-fc2e6f206c4a None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:40.428890 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e8c1da-3a3e-455f-9d3b-fc2e6f206c4a None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:40.460570 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:40.477759 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67e667a-af31-4f3f-92c4-09474a49c8e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:40.478650 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67e667a-af31-4f3f-92c4-09474a49c8e6 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:40.479083 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67e667a-af31-4f3f-92c4-09474a49c8e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:40.479506 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67e667a-af31-4f3f-92c4-09474a49c8e6 None admin] PATH_INFO: `/v3/domains` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:40.480449 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f67e667a-af31-4f3f-92c4-09474a49c8e6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:49:40.481894 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f67e667a-af31-4f3f-92c4-09474a49c8e6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:40.483635 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f67e667a-af31-4f3f-92c4-09474a49c8e6 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:40.509512 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 269/531] 10.222.0.27 () {56 vars in 1109 bytes} [Fri Aug 5 19:49:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:49:40.517072 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e78760-7c86-4724-8b10-ba40a468cbb0 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:40.522240 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67e667a-af31-4f3f-92c4-09474a49c8e6 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:40.528862 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:40.591072 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cdfcb8e-a64f-414b-8182-3a1a2893d031 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:40.591952 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdfcb8e-a64f-414b-8182-3a1a2893d031 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:40.592991 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdfcb8e-a64f-414b-8182-3a1a2893d031 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:40.593348 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdfcb8e-a64f-414b-8182-3a1a2893d031 None admin] PATH_INFO: `/v3/users` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:40.594789 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cdfcb8e-a64f-414b-8182-3a1a2893d031 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:49:40.595061 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cdfcb8e-a64f-414b-8182-3a1a2893d031 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:40.597353 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cdfcb8e-a64f-414b-8182-3a1a2893d031 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:40.638942 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 263/532] 10.222.0.27 () {56 vars in 1139 bytes} [Fri Aug 5 19:49:40 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 338 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:49:40.644727 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cdfcb8e-a64f-414b-8182-3a1a2893d031 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:40.667036 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67e667a-af31-4f3f-92c4-09474a49c8e6 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:40.706458 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced7712a-8a48-47b8-ac72-30c3fe4942fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:40.708038 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced7712a-8a48-47b8-ac72-30c3fe4942fe None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:40.708574 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced7712a-8a48-47b8-ac72-30c3fe4942fe None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:40.709044 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced7712a-8a48-47b8-ac72-30c3fe4942fe None admin] PATH_INFO: `/v3/projects/service` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:40.719168 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced7712a-8a48-47b8-ac72-30c3fe4942fe None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:40.721424 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced7712a-8a48-47b8-ac72-30c3fe4942fe None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:40.737528 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-ced7712a-8a48-47b8-ac72-30c3fe4942fe None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 19:49:40.737528 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:49:40.737528 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:49:40.737528 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:49:40.737528 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:49:40.737528 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:49:40.737528 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:49:40.737528 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:49:40.737528 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:49:40.737528 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:49:40.737528 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:49:40.737528 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:49:40.737528 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 05 19:49:40.737528 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 05 19:49:40.737528 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 05 19:49:40.738454 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 05 19:49:40.738454 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:49:40.738454 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:49:40.738454 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:49:40.738454 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:49:40.738454 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:49:40.738454 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:49:40.738454 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:49:40.738454 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Aug 05 19:49:40.738454 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:49:40.738454 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:49:40.738454 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:49:40.738454 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:49:40.738454 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:49:40.738454 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:49:40.738454 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:49:40.738454 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:49:40.739316 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 05 19:49:40.739316 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 05 19:49:40.739316 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 05 19:49:40.739316 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 05 19:49:40.739316 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 05 19:49:40.739316 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 05 19:49:40.739316 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 19:49:40.739316 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Aug 05 19:49:40.742825 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 270/533] 10.222.0.27 () {56 vars in 1111 bytes} [Fri Aug 5 19:49:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 05 19:49:40.749840 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced7712a-8a48-47b8-ac72-30c3fe4942fe None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:40.827944 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:40.859024 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cdfcb8e-a64f-414b-8182-3a1a2893d031 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:40.929803 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced7712a-8a48-47b8-ac72-30c3fe4942fe None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:40.933178 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab446ff0-f068-4d34-bb99-72be9e9a84be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:40.933764 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab446ff0-f068-4d34-bb99-72be9e9a84be None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:40.934048 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab446ff0-f068-4d34-bb99-72be9e9a84be None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:40.934327 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab446ff0-f068-4d34-bb99-72be9e9a84be None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:40.982326 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:41.006546 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9d5f5e-2fce-4b38-8971-1f86455e7256 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:41.007178 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9d5f5e-2fce-4b38-8971-1f86455e7256 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:41.007455 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9d5f5e-2fce-4b38-8971-1f86455e7256 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:41.007729 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9d5f5e-2fce-4b38-8971-1f86455e7256 None admin] PATH_INFO: `/v3/projects` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:41.008486 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d9d5f5e-2fce-4b38-8971-1f86455e7256 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:49:41.008819 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d9d5f5e-2fce-4b38-8971-1f86455e7256 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:41.010259 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d9d5f5e-2fce-4b38-8971-1f86455e7256 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:41.038985 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 271/534] 10.222.0.27 () {56 vars in 1148 bytes} [Fri Aug 5 19:49:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:49:41.039538 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab446ff0-f068-4d34-bb99-72be9e9a84be None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:41.044772 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9d5f5e-2fce-4b38-8971-1f86455e7256 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:41.048406 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab446ff0-f068-4d34-bb99-72be9e9a84be None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:41.208525 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 264/535] 10.222.0.27 () {58 vars in 1282 bytes} [Fri Aug 5 19:49:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 564 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 05 19:49:41.209427 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab446ff0-f068-4d34-bb99-72be9e9a84be None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:41.234952 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9d5f5e-2fce-4b38-8971-1f86455e7256 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:41.240703 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:41.287082 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96bbe4ca-a1d6-4c6b-bf5d-3ad924ed0ca3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:41.287692 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bbe4ca-a1d6-4c6b-bf5d-3ad924ed0ca3 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:41.287971 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bbe4ca-a1d6-4c6b-bf5d-3ad924ed0ca3 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:41.288240 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bbe4ca-a1d6-4c6b-bf5d-3ad924ed0ca3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:41.289057 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96bbe4ca-a1d6-4c6b-bf5d-3ad924ed0ca3 None admin] RBAC: Adding query filter params (role.id=6e2bc1ba438d4eb69923ef4659c181f3, scope.project.id=a70a82dcfc574fad90b3f5f00abbba58, user.id=ebf601f787694f33ab81d8021be1a2d9) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:49:41.289364 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96bbe4ca-a1d6-4c6b-bf5d-3ad924ed0ca3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:41.290965 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96bbe4ca-a1d6-4c6b-bf5d-3ad924ed0ca3 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:41.297498 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 272/536] 10.222.0.27 () {56 vars in 1374 bytes} [Fri Aug 5 19:49:41 2022] GET /identity/v3/role_assignments?user.id=ebf601f787694f33ab81d8021be1a2d9&scope.project.id=a70a82dcfc574fad90b3f5f00abbba58&role.id=6e2bc1ba438d4eb69923ef4659c181f3 => generated 259 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 19:49:41.334806 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:41.400220 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab446ff0-f068-4d34-bb99-72be9e9a84be None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:41.436638 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c758aa98-4b62-4326-b9e3-7d95c00724ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:41.437296 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c758aa98-4b62-4326-b9e3-7d95c00724ce None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:41.437574 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c758aa98-4b62-4326-b9e3-7d95c00724ce None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:41.437840 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c758aa98-4b62-4326-b9e3-7d95c00724ce None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:41.448888 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c758aa98-4b62-4326-b9e3-7d95c00724ce None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:41.450361 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c758aa98-4b62-4326-b9e3-7d95c00724ce None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:41.477513 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application [None req-c758aa98-4b62-4326-b9e3-7d95c00724ce None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:49:41.477513 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:49:41.477513 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:49:41.477513 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:49:41.477513 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:49:41.477513 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:49:41.477513 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:49:41.477513 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:49:41.477513 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:49:41.477513 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:49:41.477513 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:49:41.477513 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:49:41.477513 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 19:49:41.477513 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 19:49:41.477513 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 19:49:41.479724 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 19:49:41.479724 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:49:41.479724 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:49:41.479724 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:49:41.479724 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:49:41.479724 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:49:41.479724 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:49:41.479724 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:49:41.479724 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application with Lock( Aug 05 19:49:41.479724 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:49:41.479724 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:49:41.479724 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:49:41.479724 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:49:41.479724 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:49:41.479724 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:49:41.479724 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:49:41.479724 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:49:41.481812 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 19:49:41.481812 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 19:49:41.481812 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:49:41.481812 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Aug 05 19:49:41.481812 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 265/537] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Aug 5 19:49:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 270 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:49:41.485167 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96bbe4ca-a1d6-4c6b-bf5d-3ad924ed0ca3 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:41.605645 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96bbe4ca-a1d6-4c6b-bf5d-3ad924ed0ca3 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:41.642355 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c045a03d-f0d1-4157-b4a6-d66399554740 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:41.642974 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c045a03d-f0d1-4157-b4a6-d66399554740 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:41.643253 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c045a03d-f0d1-4157-b4a6-d66399554740 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:41.643524 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c045a03d-f0d1-4157-b4a6-d66399554740 None admin] PATH_INFO: `/v3/domains` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:41.644219 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c045a03d-f0d1-4157-b4a6-d66399554740 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:49:41.644528 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c045a03d-f0d1-4157-b4a6-d66399554740 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:41.645990 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c045a03d-f0d1-4157-b4a6-d66399554740 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:41.658687 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 273/538] 10.222.0.27 () {56 vars in 1109 bytes} [Fri Aug 5 19:49:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:49:41.663845 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c758aa98-4b62-4326-b9e3-7d95c00724ce None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:41.755500 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742c37de-65d9-4d22-8dab-281ffe7c353f None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:41.755797 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742c37de-65d9-4d22-8dab-281ffe7c353f None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:41.755991 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742c37de-65d9-4d22-8dab-281ffe7c353f None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:41.757536 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 274/539] 10.222.0.27 () {54 vars in 927 bytes} [Fri Aug 5 19:49:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 05 19:49:41.765890 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c127adf3-5d42-4b69-b6c8-256ca0a5b76b None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:41.766177 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c127adf3-5d42-4b69-b6c8-256ca0a5b76b None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:41.766462 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c127adf3-5d42-4b69-b6c8-256ca0a5b76b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:41.793423 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-c127adf3-5d42-4b69-b6c8-256ca0a5b76b None None] MFA Rules not processed for user `03d1834a64cd4a43829827cb99a72747`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:49:41.794672 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-c127adf3-5d42-4b69-b6c8-256ca0a5b76b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:49:41.823289 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c758aa98-4b62-4326-b9e3-7d95c00724ce None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:41.823804 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:41.862230 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:41.876614 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 275/540] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Aug 5 19:49:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 19:49:41.892314 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c133e796-edfd-415a-bd07-cb50dc5e7b4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:41.893292 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c133e796-edfd-415a-bd07-cb50dc5e7b4c None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:41.893754 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c133e796-edfd-415a-bd07-cb50dc5e7b4c None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:41.894225 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c133e796-edfd-415a-bd07-cb50dc5e7b4c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:41.901746 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c133e796-edfd-415a-bd07-cb50dc5e7b4c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:41.903431 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c133e796-edfd-415a-bd07-cb50dc5e7b4c None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:41.909397 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c127adf3-5d42-4b69-b6c8-256ca0a5b76b None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:41.930647 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application [None req-c133e796-edfd-415a-bd07-cb50dc5e7b4c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 19:49:41.930647 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:49:41.930647 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:49:41.930647 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:49:41.930647 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:49:41.930647 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:49:41.930647 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:49:41.930647 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:49:41.930647 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:49:41.930647 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:49:41.930647 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:49:41.930647 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:49:41.930647 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 05 19:49:41.930647 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 05 19:49:41.930647 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 05 19:49:41.931544 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 05 19:49:41.931544 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:49:41.931544 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:49:41.931544 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:49:41.931544 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:49:41.931544 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:49:41.931544 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:49:41.931544 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:49:41.931544 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application with Lock( Aug 05 19:49:41.931544 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:49:41.931544 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:49:41.931544 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:49:41.931544 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:49:41.931544 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:49:41.931544 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:49:41.931544 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:49:41.931544 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:49:41.932375 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 05 19:49:41.932375 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 05 19:49:41.932375 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 05 19:49:41.932375 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 05 19:49:41.932375 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 05 19:49:41.932375 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 05 19:49:41.932375 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 19:49:41.932375 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Aug 05 19:49:41.936013 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 266/541] 10.222.0.27 () {56 vars in 1111 bytes} [Fri Aug 5 19:49:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 05 19:49:41.943121 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c133e796-edfd-415a-bd07-cb50dc5e7b4c None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:42.098451 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c127adf3-5d42-4b69-b6c8-256ca0a5b76b None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:42.113519 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c133e796-edfd-415a-bd07-cb50dc5e7b4c None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:42.163493 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b012456b-8d7c-4612-9968-6b499719c0ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:42.164861 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b012456b-8d7c-4612-9968-6b499719c0ab None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:42.165695 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b012456b-8d7c-4612-9968-6b499719c0ab None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:42.166923 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b012456b-8d7c-4612-9968-6b499719c0ab None admin] PATH_INFO: `/v3/roles/service` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:42.167719 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24fcda6-35ef-401a-a167-d02d3f263bc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:42.168981 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24fcda6-35ef-401a-a167-d02d3f263bc8 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:42.169561 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24fcda6-35ef-401a-a167-d02d3f263bc8 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:42.171192 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24fcda6-35ef-401a-a167-d02d3f263bc8 None admin] PATH_INFO: `/v3/projects` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:42.172938 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b24fcda6-35ef-401a-a167-d02d3f263bc8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:49:42.173420 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b24fcda6-35ef-401a-a167-d02d3f263bc8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:42.177142 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b24fcda6-35ef-401a-a167-d02d3f263bc8 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:42.191761 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b012456b-8d7c-4612-9968-6b499719c0ab None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 19:49:42.196932 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-b012456b-8d7c-4612-9968-6b499719c0ab None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 05 19:49:42.196932 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:49:42.196932 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:49:42.196932 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:49:42.196932 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:49:42.196932 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:49:42.196932 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:49:42.196932 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:49:42.196932 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:49:42.196932 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:49:42.196932 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:49:42.196932 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:49:42.196932 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 19:49:42.196932 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 19:49:42.196932 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 19:49:42.196932 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 19:49:42.197972 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 19:49:42.197972 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 19:49:42.197972 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 19:49:42.197972 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 19:49:42.197972 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:49:42.197972 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:49:42.197972 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:49:42.197972 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:49:42.197972 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:49:42.197972 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:49:42.197972 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:49:42.197972 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Aug 05 19:49:42.197972 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:49:42.197972 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:49:42.197972 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:49:42.197972 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:49:42.198942 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:49:42.198942 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:49:42.198942 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:49:42.198942 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:49:42.198942 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 19:49:42.198942 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 19:49:42.198942 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 19:49:42.198942 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 19:49:42.198942 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 19:49:42.198942 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 19:49:42.198942 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 05 19:49:42.198942 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Aug 05 19:49:42.204335 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 267/543] 10.222.0.27 () {56 vars in 1148 bytes} [Fri Aug 5 19:49:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:49:42.204727 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 276/543] 10.222.0.27 () {56 vars in 1102 bytes} [Fri Aug 5 19:49:41 2022] GET /identity/v3/roles/service => generated 85 bytes in 293 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 05 19:49:42.207764 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24fcda6-35ef-401a-a167-d02d3f263bc8 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:42.210663 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b012456b-8d7c-4612-9968-6b499719c0ab None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:42.252194 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:42.288873 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:42.374409 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b012456b-8d7c-4612-9968-6b499719c0ab None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:42.374844 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24fcda6-35ef-401a-a167-d02d3f263bc8 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:42.412195 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18466625-1cc7-4b2d-8d40-aaf0848ec2bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:42.412953 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18466625-1cc7-4b2d-8d40-aaf0848ec2bf None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:42.413241 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18466625-1cc7-4b2d-8d40-aaf0848ec2bf None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:42.413505 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18466625-1cc7-4b2d-8d40-aaf0848ec2bf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:42.414472 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18466625-1cc7-4b2d-8d40-aaf0848ec2bf None admin] RBAC: Adding query filter params (role.id=6e2bc1ba438d4eb69923ef4659c181f3, scope.project.id=a70a82dcfc574fad90b3f5f00abbba58, user.id=6e632e7fe71448c29273ac65b5899aca) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:49:42.414791 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18466625-1cc7-4b2d-8d40-aaf0848ec2bf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:42.416329 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18466625-1cc7-4b2d-8d40-aaf0848ec2bf None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:42.420173 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6346da76-50c7-4343-bb72-d6ad1c8cae60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:42.421814 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6346da76-50c7-4343-bb72-d6ad1c8cae60 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:42.422290 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6346da76-50c7-4343-bb72-d6ad1c8cae60 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:42.422751 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6346da76-50c7-4343-bb72-d6ad1c8cae60 None admin] PATH_INFO: `/v3/roles` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:42.422852 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 277/544] 10.222.0.27 () {56 vars in 1374 bytes} [Fri Aug 5 19:49:42 2022] GET /identity/v3/role_assignments?user.id=6e632e7fe71448c29273ac65b5899aca&scope.project.id=a70a82dcfc574fad90b3f5f00abbba58&role.id=6e2bc1ba438d4eb69923ef4659c181f3 => generated 259 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:49:42.423832 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6346da76-50c7-4343-bb72-d6ad1c8cae60 None admin] RBAC: Adding query filter params (name=service) {{(pid=80364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:49:42.424008 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76558ae-cee3-441e-bcdc-f842acc7b199 None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:42.424333 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76558ae-cee3-441e-bcdc-f842acc7b199 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:42.424663 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76558ae-cee3-441e-bcdc-f842acc7b199 None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:42.424892 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6346da76-50c7-4343-bb72-d6ad1c8cae60 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:42.430217 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 278/545] 10.222.0.27 () {54 vars in 927 bytes} [Fri Aug 5 19:49:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:49:42.430407 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6346da76-50c7-4343-bb72-d6ad1c8cae60 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:42.435091 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367b4296-9071-441e-89fe-1632d7202498 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:42.435959 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367b4296-9071-441e-89fe-1632d7202498 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:42.436587 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367b4296-9071-441e-89fe-1632d7202498 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:42.457125 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-367b4296-9071-441e-89fe-1632d7202498 None None] MFA Rules not processed for user `03d1834a64cd4a43829827cb99a72747`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:49:42.457488 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 268/546] 10.222.0.27 () {56 vars in 1103 bytes} [Fri Aug 5 19:49:42 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:49:42.458985 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-367b4296-9071-441e-89fe-1632d7202498 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:49:42.463357 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6346da76-50c7-4343-bb72-d6ad1c8cae60 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:42.538249 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 279/547] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Aug 5 19:49:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 19:49:42.561058 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367b4296-9071-441e-89fe-1632d7202498 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:42.580397 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:42.664842 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:42.666862 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367b4296-9071-441e-89fe-1632d7202498 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:42.674711 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6346da76-50c7-4343-bb72-d6ad1c8cae60 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:42.704197 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9919a19b-af94-40a7-81d7-90496a2416fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:42.705263 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9919a19b-af94-40a7-81d7-90496a2416fc None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:42.705785 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9919a19b-af94-40a7-81d7-90496a2416fc None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:42.706260 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9919a19b-af94-40a7-81d7-90496a2416fc None admin] PATH_INFO: `/v3/roles/service` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:42.717100 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9919a19b-af94-40a7-81d7-90496a2416fc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 19:49:42.722888 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-9919a19b-af94-40a7-81d7-90496a2416fc None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 05 19:49:42.722888 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:49:42.722888 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:49:42.722888 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:49:42.722888 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:49:42.722888 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:49:42.722888 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:49:42.722888 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:49:42.722888 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:49:42.722888 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:49:42.722888 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:49:42.722888 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:49:42.722888 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 19:49:42.722888 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 19:49:42.722888 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 19:49:42.722888 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 19:49:42.723764 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 19:49:42.723764 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 19:49:42.723764 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 19:49:42.723764 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 19:49:42.723764 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:49:42.723764 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:49:42.723764 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:49:42.723764 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:49:42.723764 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:49:42.723764 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:49:42.723764 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:49:42.723764 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Aug 05 19:49:42.723764 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:49:42.723764 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:49:42.723764 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:49:42.723764 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:49:42.724570 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:49:42.724570 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:49:42.724570 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:49:42.724570 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:49:42.724570 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 19:49:42.724570 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 19:49:42.724570 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 19:49:42.724570 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 19:49:42.724570 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 19:49:42.724570 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 19:49:42.724570 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 05 19:49:42.724570 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Aug 05 19:49:42.727162 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 280/548] 10.222.0.27 () {56 vars in 1102 bytes} [Fri Aug 5 19:49:42 2022] GET /identity/v3/roles/service => generated 85 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:49:42.740521 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9919a19b-af94-40a7-81d7-90496a2416fc None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:42.754132 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018b8245-4822-4dbe-a7ce-0ffc3bf68872 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:42.754786 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018b8245-4822-4dbe-a7ce-0ffc3bf68872 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:42.755068 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018b8245-4822-4dbe-a7ce-0ffc3bf68872 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:42.755349 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018b8245-4822-4dbe-a7ce-0ffc3bf68872 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:42.788565 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-018b8245-4822-4dbe-a7ce-0ffc3bf68872 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:42.790129 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-018b8245-4822-4dbe-a7ce-0ffc3bf68872 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:42.812908 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application [None req-018b8245-4822-4dbe-a7ce-0ffc3bf68872 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:49:42.812908 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:49:42.812908 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:49:42.812908 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:49:42.812908 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:49:42.812908 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:49:42.812908 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:49:42.812908 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:49:42.812908 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:49:42.812908 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:49:42.812908 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:49:42.812908 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:49:42.812908 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 19:49:42.812908 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 19:49:42.812908 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 19:49:42.814310 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 19:49:42.814310 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:49:42.814310 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:49:42.814310 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:49:42.814310 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:49:42.814310 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:49:42.814310 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:49:42.814310 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:49:42.814310 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application with Lock( Aug 05 19:49:42.814310 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:49:42.814310 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:49:42.814310 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:49:42.814310 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:49:42.814310 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:49:42.814310 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:49:42.814310 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:49:42.814310 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:49:42.815330 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 19:49:42.815330 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 19:49:42.815330 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:49:42.815330 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Aug 05 19:49:42.815330 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 269/549] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Aug 5 19:49:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 352 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 05 19:49:42.820056 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018b8245-4822-4dbe-a7ce-0ffc3bf68872 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:42.863658 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:42.897710 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9919a19b-af94-40a7-81d7-90496a2416fc None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:42.936014 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018b8245-4822-4dbe-a7ce-0ffc3bf68872 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:42.937589 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89da75e7-d500-4b0c-b189-ff2f70e18f6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:42.938374 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89da75e7-d500-4b0c-b189-ff2f70e18f6b None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:42.938841 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89da75e7-d500-4b0c-b189-ff2f70e18f6b None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:42.939178 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89da75e7-d500-4b0c-b189-ff2f70e18f6b None admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:42.939972 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89da75e7-d500-4b0c-b189-ff2f70e18f6b None admin] RBAC: Adding query filter params (name=service) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:49:42.940344 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89da75e7-d500-4b0c-b189-ff2f70e18f6b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:42.941809 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89da75e7-d500-4b0c-b189-ff2f70e18f6b None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:42.951245 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 281/550] 10.222.0.27 () {56 vars in 1103 bytes} [Fri Aug 5 19:49:42 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:49:42.952732 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d07d1b3-fe1d-40be-9fec-593fc40e306e None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:42.953127 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d07d1b3-fe1d-40be-9fec-593fc40e306e None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:42.953470 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d07d1b3-fe1d-40be-9fec-593fc40e306e None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:42.954756 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 282/551] 10.222.0.27 () {54 vars in 927 bytes} [Fri Aug 5 19:49:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:49:42.959608 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d07d1b3-fe1d-40be-9fec-593fc40e306e None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:42.977618 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d035f6-de61-4b39-8f68-55750fd4bd8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:42.978306 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d035f6-de61-4b39-8f68-55750fd4bd8f None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:42.978588 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d035f6-de61-4b39-8f68-55750fd4bd8f None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:42.978865 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d035f6-de61-4b39-8f68-55750fd4bd8f None admin] PATH_INFO: `/v3/domains` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:42.979593 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d035f6-de61-4b39-8f68-55750fd4bd8f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:49:42.979901 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d035f6-de61-4b39-8f68-55750fd4bd8f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:42.981532 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d035f6-de61-4b39-8f68-55750fd4bd8f None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:42.995565 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 270/552] 10.222.0.27 () {56 vars in 1109 bytes} [Fri Aug 5 19:49:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 19:49:42.997343 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab9a9e7-11cf-4845-ba47-44bf03e179ca None None] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:42.997739 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab9a9e7-11cf-4845-ba47-44bf03e179ca None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:42.998027 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab9a9e7-11cf-4845-ba47-44bf03e179ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:43.018062 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.auth.core [None req-1ab9a9e7-11cf-4845-ba47-44bf03e179ca None None] MFA Rules not processed for user `03d1834a64cd4a43829827cb99a72747`. Rule list: `[]` (Enabled: `True`). {{(pid=80364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:49:43.019270 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-1ab9a9e7-11cf-4845-ba47-44bf03e179ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:49:43.022425 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:43.076169 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d07d1b3-fe1d-40be-9fec-593fc40e306e None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:43.100522 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 271/553] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Aug 5 19:49:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 19:49:43.100522 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab9a9e7-11cf-4845-ba47-44bf03e179ca None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:43.134409 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af55fd59-3dfd-4bae-a6b0-bf22a59d6a7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:43.135360 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af55fd59-3dfd-4bae-a6b0-bf22a59d6a7c None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:43.135821 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af55fd59-3dfd-4bae-a6b0-bf22a59d6a7c None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:43.136367 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af55fd59-3dfd-4bae-a6b0-bf22a59d6a7c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:43.138772 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:43.149481 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af55fd59-3dfd-4bae-a6b0-bf22a59d6a7c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:43.151463 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af55fd59-3dfd-4bae-a6b0-bf22a59d6a7c None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:43.166437 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-af55fd59-3dfd-4bae-a6b0-bf22a59d6a7c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:49:43.166437 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:49:43.166437 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:49:43.166437 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:49:43.166437 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:49:43.166437 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:49:43.166437 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:49:43.166437 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:49:43.166437 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:49:43.166437 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:49:43.166437 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:49:43.166437 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:49:43.166437 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 19:49:43.166437 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 19:49:43.166437 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 19:49:43.167441 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 19:49:43.167441 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:49:43.167441 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:49:43.167441 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:49:43.167441 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:49:43.167441 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:49:43.167441 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:49:43.167441 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:49:43.167441 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Aug 05 19:49:43.167441 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:49:43.167441 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:49:43.167441 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:49:43.167441 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:49:43.167441 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:49:43.167441 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:49:43.167441 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:49:43.167441 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:49:43.168308 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 19:49:43.168308 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 19:49:43.168308 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:49:43.168308 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Aug 05 19:49:43.171133 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 283/554] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Aug 5 19:49:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:49:43.172211 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af55fd59-3dfd-4bae-a6b0-bf22a59d6a7c None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:43.214073 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab9a9e7-11cf-4845-ba47-44bf03e179ca None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:43.253161 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f91718-6c38-43c2-a701-1fd16610fd5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:43.253810 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f91718-6c38-43c2-a701-1fd16610fd5e None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:43.254093 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f91718-6c38-43c2-a701-1fd16610fd5e None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:43.254403 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f91718-6c38-43c2-a701-1fd16610fd5e None admin] PATH_INFO: `/v3/users/swift` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:43.262681 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f91718-6c38-43c2-a701-1fd16610fd5e None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:43.264982 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f91718-6c38-43c2-a701-1fd16610fd5e None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:43.267983 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:43.280411 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application [None req-f2f91718-6c38-43c2-a701-1fd16610fd5e None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 05 19:49:43.280411 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:49:43.280411 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:49:43.280411 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:49:43.280411 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:49:43.280411 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:49:43.280411 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:49:43.280411 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:49:43.280411 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:49:43.280411 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:49:43.280411 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:49:43.280411 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:49:43.280411 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 05 19:49:43.280411 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 05 19:49:43.280411 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 05 19:49:43.280411 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 05 19:49:43.281398 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:49:43.281398 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:49:43.281398 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 05 19:49:43.281398 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 19:49:43.281398 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 05 19:49:43.281398 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 19:49:43.281398 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:49:43.281398 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:49:43.281398 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:49:43.281398 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:49:43.281398 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:49:43.281398 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application with Lock( Aug 05 19:49:43.281398 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:49:43.281398 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:49:43.281398 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:49:43.281398 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:49:43.281398 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:49:43.281398 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:49:43.282781 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:49:43.282781 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:49:43.282781 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 05 19:49:43.282781 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 05 19:49:43.282781 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 05 19:49:43.282781 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 05 19:49:43.282781 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 05 19:49:43.282781 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 05 19:49:43.282781 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 05 19:49:43.282781 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Aug 05 19:49:43.282781 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 272/555] 10.222.0.27 () {56 vars in 1096 bytes} [Fri Aug 5 19:49:43 2022] GET /identity/v3/users/swift => generated 83 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:49:43.282781 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f91718-6c38-43c2-a701-1fd16610fd5e None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:43.288572 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af55fd59-3dfd-4bae-a6b0-bf22a59d6a7c None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:43.325931 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2084fc-b2f6-4dda-8ae5-11362456938d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:43.327347 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2084fc-b2f6-4dda-8ae5-11362456938d None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:43.327835 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2084fc-b2f6-4dda-8ae5-11362456938d None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:43.328254 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2084fc-b2f6-4dda-8ae5-11362456938d None admin] PATH_INFO: `/v3/roles/service` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:43.340542 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c2084fc-b2f6-4dda-8ae5-11362456938d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 19:49:43.346185 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-1c2084fc-b2f6-4dda-8ae5-11362456938d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 05 19:49:43.346185 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:49:43.346185 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:49:43.346185 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:49:43.346185 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:49:43.346185 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:49:43.346185 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:49:43.346185 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:49:43.346185 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:49:43.346185 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:49:43.346185 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:49:43.346185 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:49:43.346185 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 19:49:43.346185 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 19:49:43.346185 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 19:49:43.346185 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 19:49:43.347294 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 19:49:43.347294 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 19:49:43.347294 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 19:49:43.347294 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 19:49:43.347294 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:49:43.347294 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:49:43.347294 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:49:43.347294 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:49:43.347294 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:49:43.347294 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:49:43.347294 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:49:43.347294 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Aug 05 19:49:43.347294 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:49:43.347294 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:49:43.347294 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:49:43.347294 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:49:43.348291 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:49:43.348291 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:49:43.348291 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:49:43.348291 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:49:43.348291 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 19:49:43.348291 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 19:49:43.348291 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 19:49:43.348291 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 19:49:43.348291 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 19:49:43.348291 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 19:49:43.348291 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 05 19:49:43.348291 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Aug 05 19:49:43.355688 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 284/556] 10.222.0.27 () {56 vars in 1102 bytes} [Fri Aug 5 19:49:43 2022] GET /identity/v3/roles/service => generated 85 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:49:43.356868 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2084fc-b2f6-4dda-8ae5-11362456938d None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:43.402307 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f91718-6c38-43c2-a701-1fd16610fd5e None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:43.409022 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:43.442616 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f586e6-198e-4fb9-a7da-49071c9fdc45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:43.443437 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f586e6-198e-4fb9-a7da-49071c9fdc45 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:43.443635 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f586e6-198e-4fb9-a7da-49071c9fdc45 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:43.443886 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f586e6-198e-4fb9-a7da-49071c9fdc45 None admin] PATH_INFO: `/v3/domains` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:43.444635 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f586e6-198e-4fb9-a7da-49071c9fdc45 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:49:43.444945 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f586e6-198e-4fb9-a7da-49071c9fdc45 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:43.446580 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f586e6-198e-4fb9-a7da-49071c9fdc45 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:43.464531 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 273/557] 10.222.0.27 () {56 vars in 1109 bytes} [Fri Aug 5 19:49:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:49:43.465481 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f586e6-198e-4fb9-a7da-49071c9fdc45 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:43.471635 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2084fc-b2f6-4dda-8ae5-11362456938d None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:43.508175 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e3787f-20ea-44cc-9089-251b72b2506e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:43.509385 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e3787f-20ea-44cc-9089-251b72b2506e None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:43.509876 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e3787f-20ea-44cc-9089-251b72b2506e None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:43.510570 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e3787f-20ea-44cc-9089-251b72b2506e None admin] PATH_INFO: `/v3/users` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:43.512311 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e3787f-20ea-44cc-9089-251b72b2506e None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:49:43.512909 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e3787f-20ea-44cc-9089-251b72b2506e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:43.514646 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e3787f-20ea-44cc-9089-251b72b2506e None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:43.532472 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:43.555210 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 285/558] 10.222.0.27 () {56 vars in 1135 bytes} [Fri Aug 5 19:49:43 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:49:43.556583 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e3787f-20ea-44cc-9089-251b72b2506e None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:43.578504 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f586e6-198e-4fb9-a7da-49071c9fdc45 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:43.617300 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7b3ce5-8382-40fa-a10a-74d383345210 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:43.617973 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7b3ce5-8382-40fa-a10a-74d383345210 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:43.618260 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7b3ce5-8382-40fa-a10a-74d383345210 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:43.618540 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7b3ce5-8382-40fa-a10a-74d383345210 None admin] PATH_INFO: `/v3/roles` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:43.619293 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a7b3ce5-8382-40fa-a10a-74d383345210 None admin] RBAC: Adding query filter params (name=service) {{(pid=80364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:49:43.619738 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a7b3ce5-8382-40fa-a10a-74d383345210 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:43.621404 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a7b3ce5-8382-40fa-a10a-74d383345210 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:43.632373 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 274/559] 10.222.0.27 () {56 vars in 1103 bytes} [Fri Aug 5 19:49:43 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:49:43.632373 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7b3ce5-8382-40fa-a10a-74d383345210 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:43.671432 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e3787f-20ea-44cc-9089-251b72b2506e None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:43.674362 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:43.723184 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25416fd8-a052-4e2a-afed-592749788308 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:43.723876 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25416fd8-a052-4e2a-afed-592749788308 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:43.724256 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25416fd8-a052-4e2a-afed-592749788308 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:43.724664 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25416fd8-a052-4e2a-afed-592749788308 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:43.734736 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25416fd8-a052-4e2a-afed-592749788308 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:43.736357 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25416fd8-a052-4e2a-afed-592749788308 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:43.748681 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-25416fd8-a052-4e2a-afed-592749788308 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 05 19:49:43.748681 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:49:43.748681 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:49:43.748681 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:49:43.748681 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:49:43.748681 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:49:43.748681 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:49:43.748681 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:49:43.748681 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:49:43.748681 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:49:43.748681 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:49:43.748681 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:49:43.748681 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 05 19:49:43.748681 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 05 19:49:43.748681 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 05 19:49:43.748681 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 05 19:49:43.749875 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:49:43.749875 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:49:43.749875 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 05 19:49:43.749875 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 19:49:43.749875 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 05 19:49:43.749875 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 19:49:43.749875 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:49:43.749875 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:49:43.749875 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:49:43.749875 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:49:43.749875 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:49:43.749875 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Aug 05 19:49:43.749875 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:49:43.749875 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:49:43.749875 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:49:43.749875 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:49:43.749875 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:49:43.749875 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:49:43.751058 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:49:43.751058 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:49:43.751058 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 05 19:49:43.751058 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 05 19:49:43.751058 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 05 19:49:43.751058 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 05 19:49:43.751058 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 05 19:49:43.751058 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 05 19:49:43.751058 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 05 19:49:43.751058 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Aug 05 19:49:43.757453 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 286/560] 10.222.0.27 () {56 vars in 1093 bytes} [Fri Aug 5 19:49:43 2022] GET /identity/v3/users/nova => generated 82 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:49:43.757453 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25416fd8-a052-4e2a-afed-592749788308 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:43.776339 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7b3ce5-8382-40fa-a10a-74d383345210 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:43.800729 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:43.815065 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e717f0-9d1c-49b8-8c86-ee03882c601a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:43.816195 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e717f0-9d1c-49b8-8c86-ee03882c601a None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:43.816757 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e717f0-9d1c-49b8-8c86-ee03882c601a None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:43.817371 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e717f0-9d1c-49b8-8c86-ee03882c601a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:43.874559 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25416fd8-a052-4e2a-afed-592749788308 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:43.892275 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e717f0-9d1c-49b8-8c86-ee03882c601a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:43.893798 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e717f0-9d1c-49b8-8c86-ee03882c601a None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:43.916292 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2288aa56-172a-4178-bb2f-2748f8419e0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:43.917249 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2288aa56-172a-4178-bb2f-2748f8419e0a None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:43.917531 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2288aa56-172a-4178-bb2f-2748f8419e0a None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:43.918127 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2288aa56-172a-4178-bb2f-2748f8419e0a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:43.929965 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2288aa56-172a-4178-bb2f-2748f8419e0a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:43.931420 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2288aa56-172a-4178-bb2f-2748f8419e0a None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:43.946752 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-2288aa56-172a-4178-bb2f-2748f8419e0a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:49:43.946752 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:49:43.946752 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:49:43.946752 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:49:43.946752 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:49:43.946752 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:49:43.946752 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:49:43.946752 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:49:43.946752 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:49:43.946752 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:49:43.946752 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:49:43.946752 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:49:43.946752 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 19:49:43.946752 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 19:49:43.946752 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 19:49:43.947774 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 19:49:43.947774 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:49:43.947774 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:49:43.947774 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:49:43.947774 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:49:43.947774 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:49:43.947774 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:49:43.947774 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:49:43.947774 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Aug 05 19:49:43.947774 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:49:43.947774 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:49:43.947774 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:49:43.947774 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:49:43.947774 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:49:43.947774 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:49:43.947774 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:49:43.947774 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:49:43.948816 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 19:49:43.948816 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 19:49:43.948816 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:49:43.948816 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Aug 05 19:49:43.948816 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 287/561] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Aug 5 19:49:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:49:43.948816 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2288aa56-172a-4178-bb2f-2748f8419e0a None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:43.967879 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:44.009577 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 275/562] 10.222.0.27 () {58 vars in 1282 bytes} [Fri Aug 5 19:49:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 05 19:49:44.014178 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492a991c-5fe5-4264-a015-98224d158490 None None] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:44.014693 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492a991c-5fe5-4264-a015-98224d158490 None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:44.015192 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492a991c-5fe5-4264-a015-98224d158490 None None] PATH_INFO: `/` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:44.020746 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 276/563] 10.222.0.27 () {54 vars in 927 bytes} [Fri Aug 5 19:49:44 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:49:44.020746 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492a991c-5fe5-4264-a015-98224d158490 None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:44.047737 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:44.077630 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2288aa56-172a-4178-bb2f-2748f8419e0a None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:44.112589 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4aec01-60a2-4a80-a7bc-600d49c589aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:44.113216 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4aec01-60a2-4a80-a7bc-600d49c589aa None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:44.113500 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4aec01-60a2-4a80-a7bc-600d49c589aa None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:44.113779 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4aec01-60a2-4a80-a7bc-600d49c589aa None admin] PATH_INFO: `/v3/users` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:44.114584 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b4aec01-60a2-4a80-a7bc-600d49c589aa None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:49:44.114895 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b4aec01-60a2-4a80-a7bc-600d49c589aa None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:44.116388 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b4aec01-60a2-4a80-a7bc-600d49c589aa None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:44.131528 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492a991c-5fe5-4264-a015-98224d158490 None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:44.135685 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 288/564] 10.222.0.27 () {56 vars in 1133 bytes} [Fri Aug 5 19:49:43 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:49:44.136577 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4aec01-60a2-4a80-a7bc-600d49c589aa None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:44.168711 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1595b99e-d20e-40b4-8579-ddfff9ccbdf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:44.169651 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1595b99e-d20e-40b4-8579-ddfff9ccbdf5 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:44.169931 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1595b99e-d20e-40b4-8579-ddfff9ccbdf5 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:44.170250 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1595b99e-d20e-40b4-8579-ddfff9ccbdf5 None admin] PATH_INFO: `/v3/domains` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:44.171077 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1595b99e-d20e-40b4-8579-ddfff9ccbdf5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:49:44.171377 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1595b99e-d20e-40b4-8579-ddfff9ccbdf5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:44.172837 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1595b99e-d20e-40b4-8579-ddfff9ccbdf5 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:44.184409 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 277/565] 10.222.0.27 () {56 vars in 1109 bytes} [Fri Aug 5 19:49:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:49:44.184409 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40eee00-a3fb-421b-8ef6-e18ff2ca6596 None None] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:44.184409 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40eee00-a3fb-421b-8ef6-e18ff2ca6596 None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:44.184409 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40eee00-a3fb-421b-8ef6-e18ff2ca6596 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:44.205361 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.auth.core [None req-b40eee00-a3fb-421b-8ef6-e18ff2ca6596 None None] MFA Rules not processed for user `03d1834a64cd4a43829827cb99a72747`. Rule list: `[]` (Enabled: `True`). {{(pid=80364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:49:44.206535 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-b40eee00-a3fb-421b-8ef6-e18ff2ca6596 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:49:44.227976 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:44.255788 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4aec01-60a2-4a80-a7bc-600d49c589aa None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:44.288593 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 278/566] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Aug 5 19:49:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 19:49:44.288593 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40eee00-a3fb-421b-8ef6-e18ff2ca6596 None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:44.300178 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e91bb9b-6d32-4ac7-8413-683361c1b5d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:44.300939 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e91bb9b-6d32-4ac7-8413-683361c1b5d9 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:44.301371 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e91bb9b-6d32-4ac7-8413-683361c1b5d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:44.301721 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e91bb9b-6d32-4ac7-8413-683361c1b5d9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:44.304134 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:44.319256 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e91bb9b-6d32-4ac7-8413-683361c1b5d9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:44.321272 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e91bb9b-6d32-4ac7-8413-683361c1b5d9 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:44.338193 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-3e91bb9b-6d32-4ac7-8413-683361c1b5d9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:49:44.338193 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:49:44.338193 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:49:44.338193 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:49:44.338193 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:49:44.338193 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:49:44.338193 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:49:44.338193 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:49:44.338193 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:49:44.338193 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:49:44.338193 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:49:44.338193 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:49:44.338193 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 19:49:44.338193 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 19:49:44.338193 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 19:49:44.339136 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 19:49:44.339136 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:49:44.339136 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:49:44.339136 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:49:44.339136 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:49:44.339136 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:49:44.339136 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:49:44.339136 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:49:44.339136 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Aug 05 19:49:44.339136 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:49:44.339136 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:49:44.339136 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:49:44.339136 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:49:44.339136 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:49:44.339136 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:49:44.339136 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:49:44.339136 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:49:44.340021 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 19:49:44.340021 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 19:49:44.340021 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:49:44.340021 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Aug 05 19:49:44.340021 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 289/567] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Aug 5 19:49:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:49:44.340823 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e91bb9b-6d32-4ac7-8413-683361c1b5d9 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:44.418391 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40eee00-a3fb-421b-8ef6-e18ff2ca6596 None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:44.449326 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e91bb9b-6d32-4ac7-8413-683361c1b5d9 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:44.455091 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30d1a19-1431-4172-9c8c-1d4cdc2d5fac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:44.455734 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30d1a19-1431-4172-9c8c-1d4cdc2d5fac None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:44.456015 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30d1a19-1431-4172-9c8c-1d4cdc2d5fac None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:44.456289 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30d1a19-1431-4172-9c8c-1d4cdc2d5fac None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:44.466701 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30d1a19-1431-4172-9c8c-1d4cdc2d5fac None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:44.468179 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30d1a19-1431-4172-9c8c-1d4cdc2d5fac None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:44.482124 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:44.483550 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application [None req-e30d1a19-1431-4172-9c8c-1d4cdc2d5fac None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:49:44.483550 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:49:44.483550 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:49:44.483550 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:49:44.483550 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:49:44.483550 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:49:44.483550 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:49:44.483550 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:49:44.483550 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:49:44.483550 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:49:44.483550 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:49:44.483550 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:49:44.483550 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 19:49:44.483550 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 19:49:44.483550 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 19:49:44.484533 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 19:49:44.484533 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:49:44.484533 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:49:44.484533 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:49:44.484533 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:49:44.484533 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:49:44.484533 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:49:44.484533 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:49:44.484533 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application with Lock( Aug 05 19:49:44.484533 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:49:44.484533 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:49:44.484533 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:49:44.484533 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:49:44.484533 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:49:44.484533 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:49:44.484533 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:49:44.484533 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:49:44.485600 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 19:49:44.485600 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 19:49:44.485600 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:49:44.485600 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Aug 05 19:49:44.489507 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 279/568] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Aug 5 19:49:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:49:44.490329 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30d1a19-1431-4172-9c8c-1d4cdc2d5fac None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:44.495361 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc81c68-d613-428d-8fc7-a1047f19a908 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:44.496174 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc81c68-d613-428d-8fc7-a1047f19a908 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:44.496643 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc81c68-d613-428d-8fc7-a1047f19a908 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:44.497108 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc81c68-d613-428d-8fc7-a1047f19a908 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:44.508178 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc81c68-d613-428d-8fc7-a1047f19a908 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:44.510528 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc81c68-d613-428d-8fc7-a1047f19a908 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:44.523973 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-7cc81c68-d613-428d-8fc7-a1047f19a908 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 05 19:49:44.523973 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:49:44.523973 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:49:44.523973 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:49:44.523973 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:49:44.523973 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:49:44.523973 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:49:44.523973 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:49:44.523973 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:49:44.523973 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:49:44.523973 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:49:44.523973 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:49:44.523973 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 05 19:49:44.523973 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 05 19:49:44.523973 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 05 19:49:44.523973 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 05 19:49:44.525357 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:49:44.525357 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:49:44.525357 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 05 19:49:44.525357 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 19:49:44.525357 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 05 19:49:44.525357 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 19:49:44.525357 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:49:44.525357 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:49:44.525357 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:49:44.525357 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:49:44.525357 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:49:44.525357 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Aug 05 19:49:44.525357 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:49:44.525357 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:49:44.525357 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:49:44.525357 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:49:44.525357 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:49:44.525357 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:49:44.526401 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:49:44.526401 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:49:44.526401 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 05 19:49:44.526401 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 05 19:49:44.526401 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 05 19:49:44.526401 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 05 19:49:44.526401 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 05 19:49:44.526401 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 05 19:49:44.526401 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 05 19:49:44.526401 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Aug 05 19:49:44.528612 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 290/569] 10.222.0.27 () {56 vars in 1099 bytes} [Fri Aug 5 19:49:44 2022] GET /identity/v3/users/cinder => generated 84 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:49:44.529546 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc81c68-d613-428d-8fc7-a1047f19a908 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:44.555488 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:44.594703 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30d1a19-1431-4172-9c8c-1d4cdc2d5fac None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:44.628132 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26570706-1a8b-4cc5-98c5-0c98d8be51dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:44.628801 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26570706-1a8b-4cc5-98c5-0c98d8be51dc None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:44.629081 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26570706-1a8b-4cc5-98c5-0c98d8be51dc None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:44.629354 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26570706-1a8b-4cc5-98c5-0c98d8be51dc None admin] PATH_INFO: `/v3/roles/service` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:44.633399 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc81c68-d613-428d-8fc7-a1047f19a908 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:44.639754 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26570706-1a8b-4cc5-98c5-0c98d8be51dc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 19:49:44.645055 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application [None req-26570706-1a8b-4cc5-98c5-0c98d8be51dc None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 05 19:49:44.645055 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:49:44.645055 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:49:44.645055 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:49:44.645055 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:49:44.645055 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:49:44.645055 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:49:44.645055 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:49:44.645055 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:49:44.645055 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:49:44.645055 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:49:44.645055 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:49:44.645055 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 19:49:44.645055 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 19:49:44.645055 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 19:49:44.645055 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 19:49:44.646198 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 19:49:44.646198 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 19:49:44.646198 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 19:49:44.646198 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 19:49:44.646198 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:49:44.646198 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:49:44.646198 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:49:44.646198 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:49:44.646198 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:49:44.646198 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:49:44.646198 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:49:44.646198 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application with Lock( Aug 05 19:49:44.646198 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:49:44.646198 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:49:44.646198 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:49:44.646198 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:49:44.647027 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:49:44.647027 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:49:44.647027 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:49:44.647027 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:49:44.647027 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 19:49:44.647027 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 19:49:44.647027 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 19:49:44.647027 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 19:49:44.647027 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 19:49:44.647027 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 19:49:44.647027 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 05 19:49:44.647027 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Aug 05 19:49:44.651526 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 280/570] 10.222.0.27 () {56 vars in 1102 bytes} [Fri Aug 5 19:49:44 2022] GET /identity/v3/roles/service => generated 85 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:49:44.652233 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26570706-1a8b-4cc5-98c5-0c98d8be51dc None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:44.666736 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10bcd577-4e2f-46e6-8d53-f826c33fe759 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:44.667348 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bcd577-4e2f-46e6-8d53-f826c33fe759 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:44.667628 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bcd577-4e2f-46e6-8d53-f826c33fe759 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:44.667898 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bcd577-4e2f-46e6-8d53-f826c33fe759 None admin] PATH_INFO: `/v3/domains` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:44.668612 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10bcd577-4e2f-46e6-8d53-f826c33fe759 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:49:44.668916 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10bcd577-4e2f-46e6-8d53-f826c33fe759 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:44.670271 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10bcd577-4e2f-46e6-8d53-f826c33fe759 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:44.682181 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 291/571] 10.222.0.27 () {56 vars in 1109 bytes} [Fri Aug 5 19:49:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:49:44.682953 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10bcd577-4e2f-46e6-8d53-f826c33fe759 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:44.727733 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:44.755840 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26570706-1a8b-4cc5-98c5-0c98d8be51dc None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:44.778360 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:44.780491 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10bcd577-4e2f-46e6-8d53-f826c33fe759 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:44.789184 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4765c31b-8c21-4a1f-a69d-58ccac0abb18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:44.789799 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4765c31b-8c21-4a1f-a69d-58ccac0abb18 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:44.790277 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4765c31b-8c21-4a1f-a69d-58ccac0abb18 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:44.790393 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4765c31b-8c21-4a1f-a69d-58ccac0abb18 None admin] PATH_INFO: `/v3/domains` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:44.791032 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4765c31b-8c21-4a1f-a69d-58ccac0abb18 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:49:44.791338 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4765c31b-8c21-4a1f-a69d-58ccac0abb18 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:44.792729 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4765c31b-8c21-4a1f-a69d-58ccac0abb18 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:44.804654 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 281/572] 10.222.0.27 () {56 vars in 1109 bytes} [Fri Aug 5 19:49:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:49:44.805873 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693466dd-4e26-4f70-ad4b-3a3216de3b57 None None] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:44.806101 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693466dd-4e26-4f70-ad4b-3a3216de3b57 None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:44.806374 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693466dd-4e26-4f70-ad4b-3a3216de3b57 None None] PATH_INFO: `/` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:44.807610 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 282/573] 10.222.0.27 () {54 vars in 927 bytes} [Fri Aug 5 19:49:44 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:49:44.808306 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693466dd-4e26-4f70-ad4b-3a3216de3b57 None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:44.817107 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ec6f8a-e0fe-437e-aa04-507298753226 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:44.817728 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ec6f8a-e0fe-437e-aa04-507298753226 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:44.818008 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ec6f8a-e0fe-437e-aa04-507298753226 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:44.818282 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ec6f8a-e0fe-437e-aa04-507298753226 None admin] PATH_INFO: `/v3/users` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:44.819077 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ec6f8a-e0fe-437e-aa04-507298753226 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:49:44.819386 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ec6f8a-e0fe-437e-aa04-507298753226 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:44.820845 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ec6f8a-e0fe-437e-aa04-507298753226 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:44.839466 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 292/574] 10.222.0.27 () {56 vars in 1137 bytes} [Fri Aug 5 19:49:44 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:49:44.840309 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ec6f8a-e0fe-437e-aa04-507298753226 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:44.917950 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693466dd-4e26-4f70-ad4b-3a3216de3b57 None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:44.939720 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ec6f8a-e0fe-437e-aa04-507298753226 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:44.942032 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:44.951129 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e6ceb7-9d05-49b0-9522-6e5bd9f63553 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:44.952300 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e6ceb7-9d05-49b0-9522-6e5bd9f63553 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:44.952619 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e6ceb7-9d05-49b0-9522-6e5bd9f63553 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:44.952891 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e6ceb7-9d05-49b0-9522-6e5bd9f63553 None admin] PATH_INFO: `/v3/roles` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:44.953601 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e6ceb7-9d05-49b0-9522-6e5bd9f63553 None admin] RBAC: Adding query filter params (name=service) {{(pid=80364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:49:44.953904 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e6ceb7-9d05-49b0-9522-6e5bd9f63553 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:44.955270 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e6ceb7-9d05-49b0-9522-6e5bd9f63553 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:44.963092 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 283/575] 10.222.0.27 () {56 vars in 1103 bytes} [Fri Aug 5 19:49:44 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:49:44.963871 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e6ceb7-9d05-49b0-9522-6e5bd9f63553 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:44.975412 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf315ce-75e6-427e-bd1d-b40c388b5ada None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:44.976055 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf315ce-75e6-427e-bd1d-b40c388b5ada None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:44.976335 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf315ce-75e6-427e-bd1d-b40c388b5ada None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:44.976640 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf315ce-75e6-427e-bd1d-b40c388b5ada None admin] PATH_INFO: `/v3/projects/service` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:44.983365 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf315ce-75e6-427e-bd1d-b40c388b5ada None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:44.984831 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf315ce-75e6-427e-bd1d-b40c388b5ada None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:44.996026 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-cbf315ce-75e6-427e-bd1d-b40c388b5ada None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 19:49:44.996026 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:49:44.996026 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:49:44.996026 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:49:44.996026 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:49:44.996026 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:49:44.996026 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:49:44.996026 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:49:44.996026 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:49:44.996026 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:49:44.996026 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:49:44.996026 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:49:44.996026 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 05 19:49:44.996026 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 05 19:49:44.996026 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 05 19:49:44.997045 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 05 19:49:44.997045 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:49:44.997045 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:49:44.997045 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:49:44.997045 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:49:44.997045 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:49:44.997045 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:49:44.997045 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:49:44.997045 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Aug 05 19:49:44.997045 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:49:44.997045 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:49:44.997045 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:49:44.997045 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:49:44.997045 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:49:44.997045 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:49:44.997045 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:49:44.997045 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:49:44.997942 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 05 19:49:44.997942 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 05 19:49:44.997942 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 05 19:49:44.997942 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 05 19:49:44.997942 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 05 19:49:44.997942 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 05 19:49:44.997942 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 19:49:44.997942 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Aug 05 19:49:44.997942 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 293/576] 10.222.0.27 () {56 vars in 1111 bytes} [Fri Aug 5 19:49:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:49:44.998414 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc7e6cf-bf40-4436-a587-f471af94f851 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:44.998980 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc7e6cf-bf40-4436-a587-f471af94f851 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:44.999498 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc7e6cf-bf40-4436-a587-f471af94f851 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:45.016262 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-4dc7e6cf-bf40-4436-a587-f471af94f851 None None] MFA Rules not processed for user `03d1834a64cd4a43829827cb99a72747`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:49:45.017058 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:45.017504 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-4dc7e6cf-bf40-4436-a587-f471af94f851 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:49:45.072234 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e6ceb7-9d05-49b0-9522-6e5bd9f63553 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:45.085709 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 294/577] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Aug 5 19:49:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 19:49:45.086547 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc7e6cf-bf40-4436-a587-f471af94f851 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:45.114208 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb26858-a5c2-4630-9eb5-b20577e70371 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:45.114872 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb26858-a5c2-4630-9eb5-b20577e70371 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:45.115153 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb26858-a5c2-4630-9eb5-b20577e70371 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:45.115424 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb26858-a5c2-4630-9eb5-b20577e70371 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:45.122127 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb26858-a5c2-4630-9eb5-b20577e70371 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:45.124101 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb26858-a5c2-4630-9eb5-b20577e70371 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:45.135222 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application [None req-ceb26858-a5c2-4630-9eb5-b20577e70371 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 19:49:45.135222 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:49:45.135222 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:49:45.135222 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:49:45.135222 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:49:45.135222 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:49:45.135222 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:49:45.135222 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:49:45.135222 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:49:45.135222 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:49:45.135222 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:49:45.135222 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:49:45.135222 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 05 19:49:45.135222 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 05 19:49:45.135222 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 05 19:49:45.136098 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 05 19:49:45.136098 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:49:45.136098 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:49:45.136098 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:49:45.136098 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:49:45.136098 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:49:45.136098 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:49:45.136098 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:49:45.136098 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application with Lock( Aug 05 19:49:45.136098 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:49:45.136098 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:49:45.136098 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:49:45.136098 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:49:45.136098 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:49:45.136098 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:49:45.136098 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:49:45.136098 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:49:45.137310 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 05 19:49:45.137310 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 05 19:49:45.137310 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 05 19:49:45.137310 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 05 19:49:45.137310 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 05 19:49:45.137310 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 05 19:49:45.137310 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 19:49:45.137310 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Aug 05 19:49:45.137310 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 284/578] 10.222.0.27 () {56 vars in 1111 bytes} [Fri Aug 5 19:49:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:49:45.137310 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb26858-a5c2-4630-9eb5-b20577e70371 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:45.180749 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:45.188606 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc7e6cf-bf40-4436-a587-f471af94f851 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:45.221716 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1736ef7b-ed72-4239-9c15-9778c826761e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:45.222359 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1736ef7b-ed72-4239-9c15-9778c826761e None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:45.222639 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1736ef7b-ed72-4239-9c15-9778c826761e None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:45.222912 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1736ef7b-ed72-4239-9c15-9778c826761e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:45.232982 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1736ef7b-ed72-4239-9c15-9778c826761e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:45.234430 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1736ef7b-ed72-4239-9c15-9778c826761e None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:45.237558 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb26858-a5c2-4630-9eb5-b20577e70371 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:45.244309 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:45.249378 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-1736ef7b-ed72-4239-9c15-9778c826761e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:49:45.249378 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:49:45.249378 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:49:45.249378 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:49:45.249378 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:49:45.249378 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:49:45.249378 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:49:45.249378 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:49:45.249378 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:49:45.249378 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:49:45.249378 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:49:45.249378 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:49:45.249378 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 19:49:45.249378 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 19:49:45.249378 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 19:49:45.250281 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 19:49:45.250281 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:49:45.250281 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:49:45.250281 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:49:45.250281 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:49:45.250281 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:49:45.250281 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:49:45.250281 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:49:45.250281 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Aug 05 19:49:45.250281 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:49:45.250281 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:49:45.250281 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:49:45.250281 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:49:45.250281 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:49:45.250281 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:49:45.250281 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:49:45.250281 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:49:45.251154 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 19:49:45.251154 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 19:49:45.251154 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:49:45.251154 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Aug 05 19:49:45.251154 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 295/579] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Aug 5 19:49:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:49:45.251512 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1736ef7b-ed72-4239-9c15-9778c826761e None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:45.277278 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72621fd-50a7-41fc-82d2-8b0330fd955d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:45.277909 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72621fd-50a7-41fc-82d2-8b0330fd955d None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:45.278187 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72621fd-50a7-41fc-82d2-8b0330fd955d None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:45.278454 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72621fd-50a7-41fc-82d2-8b0330fd955d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:45.288706 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72621fd-50a7-41fc-82d2-8b0330fd955d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:45.290152 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72621fd-50a7-41fc-82d2-8b0330fd955d None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:45.304913 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application [None req-b72621fd-50a7-41fc-82d2-8b0330fd955d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:49:45.304913 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:49:45.304913 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:49:45.304913 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:49:45.304913 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:49:45.304913 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:49:45.304913 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:49:45.304913 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:49:45.304913 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:49:45.304913 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:49:45.304913 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:49:45.304913 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:49:45.304913 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 19:49:45.304913 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 19:49:45.304913 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 19:49:45.305841 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 19:49:45.305841 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:49:45.305841 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:49:45.305841 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:49:45.305841 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:49:45.305841 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:49:45.305841 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:49:45.305841 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:49:45.305841 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application with Lock( Aug 05 19:49:45.305841 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:49:45.305841 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:49:45.305841 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:49:45.305841 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:49:45.305841 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:49:45.305841 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:49:45.305841 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:49:45.305841 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:49:45.306717 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 19:49:45.306717 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 19:49:45.306717 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:49:45.306717 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Aug 05 19:49:45.306717 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 285/580] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Aug 5 19:49:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:49:45.307067 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72621fd-50a7-41fc-82d2-8b0330fd955d None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:45.353426 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1736ef7b-ed72-4239-9c15-9778c826761e None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:45.390457 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eae9aad-b059-492b-bfb4-9504a6821b08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:45.391079 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eae9aad-b059-492b-bfb4-9504a6821b08 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:45.391357 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eae9aad-b059-492b-bfb4-9504a6821b08 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:45.391634 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eae9aad-b059-492b-bfb4-9504a6821b08 None admin] PATH_INFO: `/v3/projects` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:45.392391 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eae9aad-b059-492b-bfb4-9504a6821b08 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:49:45.392727 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eae9aad-b059-492b-bfb4-9504a6821b08 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:45.394174 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eae9aad-b059-492b-bfb4-9504a6821b08 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:45.406966 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 296/581] 10.222.0.27 () {56 vars in 1148 bytes} [Fri Aug 5 19:49:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:49:45.406966 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eae9aad-b059-492b-bfb4-9504a6821b08 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:45.416131 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:45.417212 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72621fd-50a7-41fc-82d2-8b0330fd955d None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:45.449371 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e0648a-7dc6-45e1-a8ed-b4a8e312180d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:45.450649 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e0648a-7dc6-45e1-a8ed-b4a8e312180d None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:45.450928 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e0648a-7dc6-45e1-a8ed-b4a8e312180d None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:45.451199 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e0648a-7dc6-45e1-a8ed-b4a8e312180d None admin] PATH_INFO: `/v3/roles/service` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:45.462128 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e0648a-7dc6-45e1-a8ed-b4a8e312180d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 19:49:45.468069 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application [None req-65e0648a-7dc6-45e1-a8ed-b4a8e312180d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 05 19:49:45.468069 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:49:45.468069 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:49:45.468069 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:49:45.468069 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:49:45.468069 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:49:45.468069 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:49:45.468069 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:49:45.468069 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:49:45.468069 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:49:45.468069 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:49:45.468069 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:49:45.468069 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 19:49:45.468069 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 19:49:45.468069 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 19:49:45.468069 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 19:49:45.469060 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 19:49:45.469060 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 19:49:45.469060 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 19:49:45.469060 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 19:49:45.469060 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:49:45.469060 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:49:45.469060 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:49:45.469060 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:49:45.469060 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:49:45.469060 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:49:45.469060 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:49:45.469060 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application with Lock( Aug 05 19:49:45.469060 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:49:45.469060 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:49:45.469060 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:49:45.469060 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:49:45.469897 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:49:45.469897 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:49:45.469897 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:49:45.469897 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:49:45.469897 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 19:49:45.469897 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 19:49:45.469897 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 19:49:45.469897 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 19:49:45.469897 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 19:49:45.469897 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 19:49:45.469897 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 05 19:49:45.469897 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Aug 05 19:49:45.469897 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 286/582] 10.222.0.27 () {56 vars in 1102 bytes} [Fri Aug 5 19:49:45 2022] GET /identity/v3/roles/service => generated 85 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:49:45.470567 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e0648a-7dc6-45e1-a8ed-b4a8e312180d None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:45.491990 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:45.508513 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eae9aad-b059-492b-bfb4-9504a6821b08 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:45.543452 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba736cb-a131-4c47-bdaf-e4a22213216e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:45.544061 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba736cb-a131-4c47-bdaf-e4a22213216e None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:45.544346 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba736cb-a131-4c47-bdaf-e4a22213216e None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:45.544655 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba736cb-a131-4c47-bdaf-e4a22213216e None admin] PATH_INFO: `/v3/projects` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:45.545439 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba736cb-a131-4c47-bdaf-e4a22213216e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:49:45.545745 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba736cb-a131-4c47-bdaf-e4a22213216e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:45.547177 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba736cb-a131-4c47-bdaf-e4a22213216e None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:45.558489 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 297/583] 10.222.0.27 () {56 vars in 1148 bytes} [Fri Aug 5 19:49:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:49:45.559287 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba736cb-a131-4c47-bdaf-e4a22213216e None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:45.575581 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e0648a-7dc6-45e1-a8ed-b4a8e312180d None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:45.610084 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7885d1fa-c5cb-4c42-9c0e-f08a6d6aa345 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:45.610632 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7885d1fa-c5cb-4c42-9c0e-f08a6d6aa345 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:45.610914 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7885d1fa-c5cb-4c42-9c0e-f08a6d6aa345 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:45.611189 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7885d1fa-c5cb-4c42-9c0e-f08a6d6aa345 None admin] PATH_INFO: `/v3/domains` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:45.611880 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7885d1fa-c5cb-4c42-9c0e-f08a6d6aa345 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:49:45.612190 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7885d1fa-c5cb-4c42-9c0e-f08a6d6aa345 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:45.613599 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7885d1fa-c5cb-4c42-9c0e-f08a6d6aa345 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:45.621211 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:45.627964 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 287/584] 10.222.0.27 () {56 vars in 1109 bytes} [Fri Aug 5 19:49:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:49:45.629131 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7885d1fa-c5cb-4c42-9c0e-f08a6d6aa345 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:45.657071 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba736cb-a131-4c47-bdaf-e4a22213216e None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:45.689974 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e9d3c3-ad13-451e-b966-90c31164c72d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:45.690586 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e9d3c3-ad13-451e-b966-90c31164c72d None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:45.690874 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e9d3c3-ad13-451e-b966-90c31164c72d None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:45.691142 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e9d3c3-ad13-451e-b966-90c31164c72d None admin] PATH_INFO: `/v3/domains` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:45.691831 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e9d3c3-ad13-451e-b966-90c31164c72d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:49:45.692135 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e9d3c3-ad13-451e-b966-90c31164c72d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:45.693516 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e9d3c3-ad13-451e-b966-90c31164c72d None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:45.709190 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 298/585] 10.222.0.27 () {56 vars in 1109 bytes} [Fri Aug 5 19:49:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:49:45.709190 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e9d3c3-ad13-451e-b966-90c31164c72d None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:45.710622 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:45.731862 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7885d1fa-c5cb-4c42-9c0e-f08a6d6aa345 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:45.766034 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473c1a03-8b2c-47dc-ba9a-0c7e080be9ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:45.766634 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473c1a03-8b2c-47dc-ba9a-0c7e080be9ac None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:45.766918 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473c1a03-8b2c-47dc-ba9a-0c7e080be9ac None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:45.767198 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473c1a03-8b2c-47dc-ba9a-0c7e080be9ac None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:45.767959 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473c1a03-8b2c-47dc-ba9a-0c7e080be9ac None admin] RBAC: Adding query filter params (role.id=6e2bc1ba438d4eb69923ef4659c181f3, scope.project.id=a70a82dcfc574fad90b3f5f00abbba58, user.id=6533931da2d34447881008b43372b0d5) {{(pid=80364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:49:45.768264 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473c1a03-8b2c-47dc-ba9a-0c7e080be9ac None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:45.769745 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473c1a03-8b2c-47dc-ba9a-0c7e080be9ac None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:45.775298 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 288/586] 10.222.0.27 () {56 vars in 1374 bytes} [Fri Aug 5 19:49:45 2022] GET /identity/v3/role_assignments?user.id=6533931da2d34447881008b43372b0d5&scope.project.id=a70a82dcfc574fad90b3f5f00abbba58&role.id=6e2bc1ba438d4eb69923ef4659c181f3 => generated 259 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:49:45.776047 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473c1a03-8b2c-47dc-ba9a-0c7e080be9ac None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:45.807794 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e9d3c3-ad13-451e-b966-90c31164c72d None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:45.838768 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:45.851179 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034c33e9-d711-417f-9677-5d2df874ad45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:45.851921 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034c33e9-d711-417f-9677-5d2df874ad45 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:45.852226 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034c33e9-d711-417f-9677-5d2df874ad45 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:45.852606 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034c33e9-d711-417f-9677-5d2df874ad45 None admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:45.853333 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034c33e9-d711-417f-9677-5d2df874ad45 None admin] RBAC: Adding query filter params (name=service) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:49:45.853638 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034c33e9-d711-417f-9677-5d2df874ad45 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:45.855142 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034c33e9-d711-417f-9677-5d2df874ad45 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:45.866317 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 299/587] 10.222.0.27 () {56 vars in 1103 bytes} [Fri Aug 5 19:49:45 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:49:45.866317 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034c33e9-d711-417f-9677-5d2df874ad45 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:45.897661 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473c1a03-8b2c-47dc-ba9a-0c7e080be9ac None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:45.936824 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07235079-669b-4cd3-8014-cd7520257d98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:45.937496 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07235079-669b-4cd3-8014-cd7520257d98 None admin] REQUEST_METHOD: `PUT` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:45.937779 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07235079-669b-4cd3-8014-cd7520257d98 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:45.938050 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07235079-669b-4cd3-8014-cd7520257d98 None admin] PATH_INFO: `/v3/projects/a70a82dcfc574fad90b3f5f00abbba58/users/11001a2ad96f4ec8a35aab2c64376875/roles/6e2bc1ba438d4eb69923ef4659c181f3` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:45.950368 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:45.954037 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07235079-669b-4cd3-8014-cd7520257d98 None admin] RBAC: Authorizing `identity:create_grant(project_id=a70a82dcfc574fad90b3f5f00abbba58, user_id=11001a2ad96f4ec8a35aab2c64376875, role_id=6e2bc1ba438d4eb69923ef4659c181f3)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:45.955734 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07235079-669b-4cd3-8014-cd7520257d98 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:45.970457 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 289/588] 10.222.0.27 () {58 vars in 1439 bytes} [Fri Aug 5 19:49:45 2022] PUT /identity/v3/projects/a70a82dcfc574fad90b3f5f00abbba58/users/11001a2ad96f4ec8a35aab2c64376875/roles/6e2bc1ba438d4eb69923ef4659c181f3 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 19:49:45.971163 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07235079-669b-4cd3-8014-cd7520257d98 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:46.020318 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034c33e9-d711-417f-9677-5d2df874ad45 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:46.075393 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f2eb97-c18d-4661-aec3-c5f55a1b3c61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:46.076130 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f2eb97-c18d-4661-aec3-c5f55a1b3c61 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:46.076421 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f2eb97-c18d-4661-aec3-c5f55a1b3c61 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:46.077291 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f2eb97-c18d-4661-aec3-c5f55a1b3c61 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:46.088038 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f2eb97-c18d-4661-aec3-c5f55a1b3c61 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:46.090291 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f2eb97-c18d-4661-aec3-c5f55a1b3c61 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:46.096837 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07235079-669b-4cd3-8014-cd7520257d98 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:46.110743 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-97f2eb97-c18d-4661-aec3-c5f55a1b3c61 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 19:49:46.110743 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:49:46.110743 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:49:46.110743 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:49:46.110743 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:49:46.110743 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:49:46.110743 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:49:46.110743 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:49:46.110743 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:49:46.110743 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:49:46.110743 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:49:46.110743 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:49:46.110743 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 05 19:49:46.110743 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 05 19:49:46.110743 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 05 19:49:46.111680 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 05 19:49:46.111680 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:49:46.111680 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:49:46.111680 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:49:46.111680 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:49:46.111680 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:49:46.111680 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:49:46.111680 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:49:46.111680 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Aug 05 19:49:46.111680 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:49:46.111680 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:49:46.111680 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:49:46.111680 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:49:46.111680 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:49:46.111680 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:49:46.111680 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:49:46.111680 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:49:46.112588 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 05 19:49:46.112588 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 05 19:49:46.112588 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 05 19:49:46.112588 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 05 19:49:46.112588 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 05 19:49:46.112588 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 05 19:49:46.112588 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 19:49:46.112588 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Aug 05 19:49:46.112588 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 300/589] 10.222.0.27 () {56 vars in 1111 bytes} [Fri Aug 5 19:49:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:49:46.113604 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f2eb97-c18d-4661-aec3-c5f55a1b3c61 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:46.147435 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2713eda-1bd4-4624-9f21-a805ce1cd06c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:46.148402 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2713eda-1bd4-4624-9f21-a805ce1cd06c None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:46.148770 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2713eda-1bd4-4624-9f21-a805ce1cd06c None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:46.149105 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2713eda-1bd4-4624-9f21-a805ce1cd06c None admin] PATH_INFO: `/v3/users/glance` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:46.157847 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2713eda-1bd4-4624-9f21-a805ce1cd06c None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:46.159282 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2713eda-1bd4-4624-9f21-a805ce1cd06c None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:46.171768 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application [None req-b2713eda-1bd4-4624-9f21-a805ce1cd06c None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 05 19:49:46.171768 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:49:46.171768 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:49:46.171768 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:49:46.171768 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:49:46.171768 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:49:46.171768 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:49:46.171768 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:49:46.171768 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:49:46.171768 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:49:46.171768 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:49:46.171768 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:49:46.171768 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 05 19:49:46.171768 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 05 19:49:46.171768 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 05 19:49:46.171768 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 05 19:49:46.172773 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:49:46.172773 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:49:46.172773 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 05 19:49:46.172773 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 19:49:46.172773 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 05 19:49:46.172773 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 19:49:46.172773 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:49:46.172773 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:49:46.172773 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:49:46.172773 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:49:46.172773 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:49:46.172773 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application with Lock( Aug 05 19:49:46.172773 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:49:46.172773 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:49:46.172773 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:49:46.172773 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:49:46.172773 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:49:46.172773 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:49:46.173693 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:49:46.173693 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:49:46.173693 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 05 19:49:46.173693 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 05 19:49:46.173693 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 05 19:49:46.173693 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 05 19:49:46.173693 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 05 19:49:46.173693 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 05 19:49:46.173693 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 05 19:49:46.173693 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Aug 05 19:49:46.173693 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 290/590] 10.222.0.27 () {56 vars in 1099 bytes} [Fri Aug 5 19:49:45 2022] GET /identity/v3/users/glance => generated 84 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:49:46.174288 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2713eda-1bd4-4624-9f21-a805ce1cd06c None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:46.184636 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:46.231261 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:46.242121 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f2eb97-c18d-4661-aec3-c5f55a1b3c61 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:46.285308 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2713eda-1bd4-4624-9f21-a805ce1cd06c None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:46.285543 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c49633e-fef2-42b1-9038-17db16583f6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:46.286038 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c49633e-fef2-42b1-9038-17db16583f6b None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:46.286325 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c49633e-fef2-42b1-9038-17db16583f6b None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:46.286603 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c49633e-fef2-42b1-9038-17db16583f6b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:46.302209 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c49633e-fef2-42b1-9038-17db16583f6b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:46.304312 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c49633e-fef2-42b1-9038-17db16583f6b None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:46.322613 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-5c49633e-fef2-42b1-9038-17db16583f6b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:49:46.322613 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:49:46.322613 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:49:46.322613 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:49:46.322613 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:49:46.322613 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:49:46.322613 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:49:46.322613 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:49:46.322613 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:49:46.322613 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:49:46.322613 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:49:46.322613 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:49:46.322613 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 19:49:46.322613 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 19:49:46.322613 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 19:49:46.323509 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 19:49:46.323509 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:49:46.323509 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:49:46.323509 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:49:46.323509 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:49:46.323509 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:49:46.323509 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:49:46.323509 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:49:46.323509 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Aug 05 19:49:46.323509 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:49:46.323509 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:49:46.323509 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:49:46.323509 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:49:46.323509 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:49:46.323509 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:49:46.323509 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:49:46.323509 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:49:46.324376 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 19:49:46.324376 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 19:49:46.324376 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:49:46.324376 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Aug 05 19:49:46.324376 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 301/591] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Aug 5 19:49:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:49:46.325367 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c49633e-fef2-42b1-9038-17db16583f6b None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:46.325470 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9894517a-513d-4216-8799-b012051663e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:46.326072 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9894517a-513d-4216-8799-b012051663e6 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:46.328347 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9894517a-513d-4216-8799-b012051663e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:46.328347 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9894517a-513d-4216-8799-b012051663e6 None admin] PATH_INFO: `/v3/projects` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:46.328347 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9894517a-513d-4216-8799-b012051663e6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:49:46.328347 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9894517a-513d-4216-8799-b012051663e6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:46.329735 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9894517a-513d-4216-8799-b012051663e6 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:46.350819 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 291/592] 10.222.0.27 () {56 vars in 1148 bytes} [Fri Aug 5 19:49:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:49:46.351597 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9894517a-513d-4216-8799-b012051663e6 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:46.454835 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:46.457865 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c49633e-fef2-42b1-9038-17db16583f6b None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:46.491809 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9894517a-513d-4216-8799-b012051663e6 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:46.505836 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8ef807-9d3e-465b-bb10-b12dea01845b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:46.506827 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8ef807-9d3e-465b-bb10-b12dea01845b None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:46.507422 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:46.507910 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8ef807-9d3e-465b-bb10-b12dea01845b None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:46.509422 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8ef807-9d3e-465b-bb10-b12dea01845b None admin] PATH_INFO: `/v3/users` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:46.510978 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc8ef807-9d3e-465b-bb10-b12dea01845b None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:49:46.512154 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc8ef807-9d3e-465b-bb10-b12dea01845b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:46.515184 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc8ef807-9d3e-465b-bb10-b12dea01845b None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:46.533169 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e13e1dc-9c1f-4065-bfa8-bf62d2d08816 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:46.533786 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e13e1dc-9c1f-4065-bfa8-bf62d2d08816 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:46.534063 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e13e1dc-9c1f-4065-bfa8-bf62d2d08816 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:46.534332 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e13e1dc-9c1f-4065-bfa8-bf62d2d08816 None admin] PATH_INFO: `/v3/domains` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:46.535027 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e13e1dc-9c1f-4065-bfa8-bf62d2d08816 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:49:46.535331 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e13e1dc-9c1f-4065-bfa8-bf62d2d08816 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:46.536856 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e13e1dc-9c1f-4065-bfa8-bf62d2d08816 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:46.545484 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 302/593] 10.222.0.27 () {56 vars in 1137 bytes} [Fri Aug 5 19:49:46 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:49:46.545677 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8ef807-9d3e-465b-bb10-b12dea01845b None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:46.551612 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 292/594] 10.222.0.27 () {56 vars in 1109 bytes} [Fri Aug 5 19:49:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:49:46.552770 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e13e1dc-9c1f-4065-bfa8-bf62d2d08816 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:46.657286 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e13e1dc-9c1f-4065-bfa8-bf62d2d08816 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:46.658954 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8ef807-9d3e-465b-bb10-b12dea01845b None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:46.693153 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2292357c-3cb5-4307-9ac9-0697b2995476 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:46.694156 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2292357c-3cb5-4307-9ac9-0697b2995476 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:46.694634 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2292357c-3cb5-4307-9ac9-0697b2995476 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:46.695011 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a6e052-b7bf-4d16-ad1f-afa372d5dadb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:46.695360 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2292357c-3cb5-4307-9ac9-0697b2995476 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:46.695677 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a6e052-b7bf-4d16-ad1f-afa372d5dadb None admin] REQUEST_METHOD: `PUT` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:46.695951 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a6e052-b7bf-4d16-ad1f-afa372d5dadb None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:46.696337 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a6e052-b7bf-4d16-ad1f-afa372d5dadb None admin] PATH_INFO: `/v3/projects/a70a82dcfc574fad90b3f5f00abbba58/users/c092a23e14e24265973ff5e899670fc8/roles/6e2bc1ba438d4eb69923ef4659c181f3` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:46.707387 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2292357c-3cb5-4307-9ac9-0697b2995476 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:46.709162 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2292357c-3cb5-4307-9ac9-0697b2995476 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:46.716806 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a6e052-b7bf-4d16-ad1f-afa372d5dadb None admin] RBAC: Authorizing `identity:create_grant(project_id=a70a82dcfc574fad90b3f5f00abbba58, user_id=c092a23e14e24265973ff5e899670fc8, role_id=6e2bc1ba438d4eb69923ef4659c181f3)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:46.718903 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a6e052-b7bf-4d16-ad1f-afa372d5dadb None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:46.725557 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application [None req-2292357c-3cb5-4307-9ac9-0697b2995476 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:49:46.725557 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:49:46.725557 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:49:46.725557 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:49:46.725557 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:49:46.725557 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:49:46.725557 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:49:46.725557 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:49:46.725557 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:49:46.725557 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:49:46.725557 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:49:46.725557 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:49:46.725557 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 19:49:46.725557 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 19:49:46.725557 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 19:49:46.727002 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 19:49:46.727002 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:49:46.727002 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:49:46.727002 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:49:46.727002 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:49:46.727002 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:49:46.727002 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:49:46.727002 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:49:46.727002 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application with Lock( Aug 05 19:49:46.727002 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:49:46.727002 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:49:46.727002 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:49:46.727002 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:49:46.727002 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:49:46.727002 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:49:46.727002 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:49:46.727002 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:49:46.728115 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 19:49:46.728115 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 19:49:46.728115 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:49:46.728115 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Aug 05 19:49:46.734261 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 293/595] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Aug 5 19:49:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:49:46.735211 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2292357c-3cb5-4307-9ac9-0697b2995476 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:46.737333 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 303/596] 10.222.0.27 () {58 vars in 1439 bytes} [Fri Aug 5 19:49:46 2022] PUT /identity/v3/projects/a70a82dcfc574fad90b3f5f00abbba58/users/c092a23e14e24265973ff5e899670fc8/roles/6e2bc1ba438d4eb69923ef4659c181f3 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 19:49:46.738590 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a6e052-b7bf-4d16-ad1f-afa372d5dadb None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:46.760631 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:46.793130 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:46.861862 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a6e052-b7bf-4d16-ad1f-afa372d5dadb None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:46.902020 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0444b910-a0b1-4fef-8816-92ecf09742ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:46.902929 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0444b910-a0b1-4fef-8816-92ecf09742ec None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:46.903365 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0444b910-a0b1-4fef-8816-92ecf09742ec None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:46.903799 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0444b910-a0b1-4fef-8816-92ecf09742ec None admin] PATH_INFO: `/v3/domains` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:46.904844 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0444b910-a0b1-4fef-8816-92ecf09742ec None admin] RBAC: Adding query filter params (name=Default) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:49:46.905363 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0444b910-a0b1-4fef-8816-92ecf09742ec None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:46.906887 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0444b910-a0b1-4fef-8816-92ecf09742ec None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:46.921495 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 304/597] 10.222.0.27 () {56 vars in 1109 bytes} [Fri Aug 5 19:49:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:49:46.929865 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0444b910-a0b1-4fef-8816-92ecf09742ec None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:46.945145 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2292357c-3cb5-4307-9ac9-0697b2995476 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:47.015376 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf51cd8-930b-4852-a394-dc1809e2546d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:47.016106 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf51cd8-930b-4852-a394-dc1809e2546d None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:47.016479 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf51cd8-930b-4852-a394-dc1809e2546d None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:47.016872 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf51cd8-930b-4852-a394-dc1809e2546d None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:47.031097 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf51cd8-930b-4852-a394-dc1809e2546d None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:47.035599 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf51cd8-930b-4852-a394-dc1809e2546d None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:47.053224 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application [None req-2bf51cd8-930b-4852-a394-dc1809e2546d None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 05 19:49:47.053224 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:49:47.053224 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:49:47.053224 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:49:47.053224 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:49:47.053224 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:49:47.053224 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:49:47.053224 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:49:47.053224 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:49:47.053224 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:49:47.053224 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:49:47.053224 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:49:47.053224 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 05 19:49:47.053224 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 05 19:49:47.053224 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 05 19:49:47.053224 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 05 19:49:47.054470 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:49:47.054470 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:49:47.054470 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 05 19:49:47.054470 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 19:49:47.054470 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 05 19:49:47.054470 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 19:49:47.054470 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:49:47.054470 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:49:47.054470 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:49:47.054470 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:49:47.054470 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:49:47.054470 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application with Lock( Aug 05 19:49:47.054470 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:49:47.054470 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:49:47.054470 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:49:47.054470 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:49:47.054470 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:49:47.054470 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:49:47.055838 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:49:47.055838 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:49:47.055838 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 05 19:49:47.055838 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 05 19:49:47.055838 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 05 19:49:47.055838 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 05 19:49:47.055838 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 05 19:49:47.055838 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 05 19:49:47.055838 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 05 19:49:47.055838 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Aug 05 19:49:47.057905 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 294/598] 10.222.0.27 () {56 vars in 1102 bytes} [Fri Aug 5 19:49:46 2022] GET /identity/v3/users/neutron => generated 85 bytes in 323 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:49:47.060271 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:47.065701 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf51cd8-930b-4852-a394-dc1809e2546d None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:47.106781 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0444b910-a0b1-4fef-8816-92ecf09742ec None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:47.143441 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa0f560-f8a8-4576-95fb-5e893a278bf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:47.144081 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa0f560-f8a8-4576-95fb-5e893a278bf2 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:47.144361 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa0f560-f8a8-4576-95fb-5e893a278bf2 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:47.144679 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa0f560-f8a8-4576-95fb-5e893a278bf2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:47.152092 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa0f560-f8a8-4576-95fb-5e893a278bf2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:47.157731 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa0f560-f8a8-4576-95fb-5e893a278bf2 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:47.161643 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:47.171086 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-1fa0f560-f8a8-4576-95fb-5e893a278bf2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 19:49:47.171086 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:49:47.171086 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:49:47.171086 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:49:47.171086 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:49:47.171086 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:49:47.171086 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:49:47.171086 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:49:47.171086 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:49:47.171086 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:49:47.171086 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:49:47.171086 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:49:47.171086 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 05 19:49:47.171086 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 05 19:49:47.171086 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 05 19:49:47.171995 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 05 19:49:47.171995 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:49:47.171995 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:49:47.171995 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:49:47.171995 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:49:47.171995 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:49:47.171995 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:49:47.171995 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:49:47.171995 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Aug 05 19:49:47.171995 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:49:47.171995 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:49:47.171995 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:49:47.171995 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:49:47.171995 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:49:47.171995 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:49:47.171995 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:49:47.171995 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:49:47.172908 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 05 19:49:47.172908 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 05 19:49:47.172908 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 05 19:49:47.172908 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 05 19:49:47.172908 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 05 19:49:47.172908 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 05 19:49:47.172908 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 19:49:47.172908 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Aug 05 19:49:47.172908 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 305/599] 10.222.0.27 () {56 vars in 1111 bytes} [Fri Aug 5 19:49:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:49:47.181472 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa0f560-f8a8-4576-95fb-5e893a278bf2 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:47.214715 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf51cd8-930b-4852-a394-dc1809e2546d None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:47.258462 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27fd836b-df9f-40ca-8ffd-a2db3091f9e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:47.259203 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fd836b-df9f-40ca-8ffd-a2db3091f9e2 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:47.259482 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fd836b-df9f-40ca-8ffd-a2db3091f9e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:47.259755 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fd836b-df9f-40ca-8ffd-a2db3091f9e2 None admin] PATH_INFO: `/v3/users` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:47.260584 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27fd836b-df9f-40ca-8ffd-a2db3091f9e2 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:49:47.260893 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27fd836b-df9f-40ca-8ffd-a2db3091f9e2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:47.262328 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27fd836b-df9f-40ca-8ffd-a2db3091f9e2 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:47.301432 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 295/600] 10.222.0.27 () {56 vars in 1139 bytes} [Fri Aug 5 19:49:47 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:49:47.309531 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27fd836b-df9f-40ca-8ffd-a2db3091f9e2 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:47.357259 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa0f560-f8a8-4576-95fb-5e893a278bf2 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:47.429258 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:47.435069 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baad7f2d-3e4b-457b-b7cb-f5a39336bf79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:47.437166 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baad7f2d-3e4b-457b-b7cb-f5a39336bf79 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:47.437461 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baad7f2d-3e4b-457b-b7cb-f5a39336bf79 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:47.437736 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baad7f2d-3e4b-457b-b7cb-f5a39336bf79 None admin] PATH_INFO: `/v3/projects` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:47.438649 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baad7f2d-3e4b-457b-b7cb-f5a39336bf79 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:49:47.439089 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baad7f2d-3e4b-457b-b7cb-f5a39336bf79 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:47.440827 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baad7f2d-3e4b-457b-b7cb-f5a39336bf79 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:47.455258 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 306/601] 10.222.0.27 () {56 vars in 1148 bytes} [Fri Aug 5 19:49:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:49:47.464817 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baad7f2d-3e4b-457b-b7cb-f5a39336bf79 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:47.538614 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27fd836b-df9f-40ca-8ffd-a2db3091f9e2 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:47.593631 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baad7f2d-3e4b-457b-b7cb-f5a39336bf79 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:47.600111 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76e8dc6-e0a5-4c18-a89f-fc1c411ede0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:47.601001 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76e8dc6-e0a5-4c18-a89f-fc1c411ede0a None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:47.601440 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76e8dc6-e0a5-4c18-a89f-fc1c411ede0a None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:47.608914 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76e8dc6-e0a5-4c18-a89f-fc1c411ede0a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:47.629592 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a76e8dc6-e0a5-4c18-a89f-fc1c411ede0a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:47.631314 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a76e8dc6-e0a5-4c18-a89f-fc1c411ede0a None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:47.636983 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:47.643477 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0443377-023b-46d2-9c84-c7d5c40a1ab7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:47.644158 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0443377-023b-46d2-9c84-c7d5c40a1ab7 None admin] REQUEST_METHOD: `PUT` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:47.644437 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0443377-023b-46d2-9c84-c7d5c40a1ab7 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:47.644734 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0443377-023b-46d2-9c84-c7d5c40a1ab7 None admin] PATH_INFO: `/v3/projects/a70a82dcfc574fad90b3f5f00abbba58/users/ebf601f787694f33ab81d8021be1a2d9/roles/6e2bc1ba438d4eb69923ef4659c181f3` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:47.663322 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application [None req-a76e8dc6-e0a5-4c18-a89f-fc1c411ede0a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:49:47.663322 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:49:47.663322 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:49:47.663322 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:49:47.663322 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:49:47.663322 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:49:47.663322 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:49:47.663322 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:49:47.663322 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:49:47.663322 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:49:47.663322 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:49:47.663322 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:49:47.663322 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 19:49:47.663322 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 19:49:47.663322 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 19:49:47.664453 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 19:49:47.664453 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:49:47.664453 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:49:47.664453 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:49:47.664453 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:49:47.664453 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:49:47.664453 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:49:47.664453 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:49:47.664453 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application with Lock( Aug 05 19:49:47.664453 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:49:47.664453 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:49:47.664453 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:49:47.664453 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:49:47.664453 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:49:47.664453 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:49:47.664453 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:49:47.664453 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:49:47.665605 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 19:49:47.665605 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 19:49:47.665605 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:49:47.665605 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Aug 05 19:49:47.665605 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 296/602] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Aug 5 19:49:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 355 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:49:47.671169 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76e8dc6-e0a5-4c18-a89f-fc1c411ede0a None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:47.674261 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0443377-023b-46d2-9c84-c7d5c40a1ab7 None admin] RBAC: Authorizing `identity:create_grant(project_id=a70a82dcfc574fad90b3f5f00abbba58, user_id=ebf601f787694f33ab81d8021be1a2d9, role_id=6e2bc1ba438d4eb69923ef4659c181f3)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:47.676605 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0443377-023b-46d2-9c84-c7d5c40a1ab7 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:47.699590 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 307/603] 10.222.0.27 () {58 vars in 1439 bytes} [Fri Aug 5 19:49:47 2022] PUT /identity/v3/projects/a70a82dcfc574fad90b3f5f00abbba58/users/ebf601f787694f33ab81d8021be1a2d9/roles/6e2bc1ba438d4eb69923ef4659c181f3 => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 19:49:47.825859 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76e8dc6-e0a5-4c18-a89f-fc1c411ede0a None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:47.872374 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e16d70a-ac1f-4a0e-b6e3-16eb7dfb2aa2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:47.874350 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e16d70a-ac1f-4a0e-b6e3-16eb7dfb2aa2 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:47.874350 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e16d70a-ac1f-4a0e-b6e3-16eb7dfb2aa2 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:47.874350 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e16d70a-ac1f-4a0e-b6e3-16eb7dfb2aa2 None admin] PATH_INFO: `/v3/domains` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:47.874350 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e16d70a-ac1f-4a0e-b6e3-16eb7dfb2aa2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:49:47.874669 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e16d70a-ac1f-4a0e-b6e3-16eb7dfb2aa2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:47.876515 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e16d70a-ac1f-4a0e-b6e3-16eb7dfb2aa2 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:47.889831 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 297/604] 10.222.0.27 () {56 vars in 1109 bytes} [Fri Aug 5 19:49:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:49:47.895254 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0443377-023b-46d2-9c84-c7d5c40a1ab7 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:47.945975 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:47.992539 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05e5e0c-9a02-45f0-8e71-1ee2b940e213 None None] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:47.993424 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05e5e0c-9a02-45f0-8e71-1ee2b940e213 None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:47.994147 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05e5e0c-9a02-45f0-8e71-1ee2b940e213 None None] PATH_INFO: `/` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:48.000876 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 298/605] 10.222.0.27 () {54 vars in 927 bytes} [Fri Aug 5 19:49:47 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:49:48.003689 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9ec9aa-4a74-4a71-82e9-5727df685042 None None] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:48.004251 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9ec9aa-4a74-4a71-82e9-5727df685042 None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:48.004763 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9ec9aa-4a74-4a71-82e9-5727df685042 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:48.024165 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0443377-023b-46d2-9c84-c7d5c40a1ab7 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:48.047010 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.auth.core [None req-1b9ec9aa-4a74-4a71-82e9-5727df685042 None None] MFA Rules not processed for user `03d1834a64cd4a43829827cb99a72747`. Rule list: `[]` (Enabled: `True`). {{(pid=80364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:49:48.048597 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-1b9ec9aa-4a74-4a71-82e9-5727df685042 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:49:48.061861 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2c9f28-4dbd-4db1-bc37-4ea4317d4cf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:48.062558 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2c9f28-4dbd-4db1-bc37-4ea4317d4cf7 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:48.062838 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2c9f28-4dbd-4db1-bc37-4ea4317d4cf7 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:48.063119 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2c9f28-4dbd-4db1-bc37-4ea4317d4cf7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:48.070459 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b2c9f28-4dbd-4db1-bc37-4ea4317d4cf7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:48.072008 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b2c9f28-4dbd-4db1-bc37-4ea4317d4cf7 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:48.085042 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-0b2c9f28-4dbd-4db1-bc37-4ea4317d4cf7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 19:49:48.085042 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:49:48.085042 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:49:48.085042 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:49:48.085042 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:49:48.085042 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:49:48.085042 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:49:48.085042 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:49:48.085042 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:49:48.085042 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:49:48.085042 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:49:48.085042 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:49:48.085042 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 05 19:49:48.085042 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 05 19:49:48.085042 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 05 19:49:48.085964 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 05 19:49:48.085964 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:49:48.085964 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:49:48.085964 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:49:48.085964 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:49:48.085964 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:49:48.085964 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:49:48.085964 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:49:48.085964 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Aug 05 19:49:48.085964 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:49:48.085964 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:49:48.085964 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:49:48.085964 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:49:48.085964 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:49:48.085964 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:49:48.085964 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:49:48.085964 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:49:48.086872 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 05 19:49:48.086872 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 05 19:49:48.086872 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 05 19:49:48.086872 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 05 19:49:48.086872 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 05 19:49:48.086872 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 05 19:49:48.086872 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 19:49:48.086872 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Aug 05 19:49:48.086872 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 308/606] 10.222.0.27 () {56 vars in 1111 bytes} [Fri Aug 5 19:49:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:49:48.092368 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2c9f28-4dbd-4db1-bc37-4ea4317d4cf7 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:48.135892 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:48.158521 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 299/607] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Aug 5 19:49:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 05 19:49:48.193361 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9ec9aa-4a74-4a71-82e9-5727df685042 None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:48.306397 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9ec9aa-4a74-4a71-82e9-5727df685042 None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:48.312437 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:48.319204 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2c9f28-4dbd-4db1-bc37-4ea4317d4cf7 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:48.346911 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc56c1fd-36ed-4662-b62c-953465e0c578 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:48.347617 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc56c1fd-36ed-4662-b62c-953465e0c578 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:48.347843 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc56c1fd-36ed-4662-b62c-953465e0c578 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:48.348116 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc56c1fd-36ed-4662-b62c-953465e0c578 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:48.367304 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6960a0-8f98-4604-b24d-25da631b4ee1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:48.367522 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc56c1fd-36ed-4662-b62c-953465e0c578 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 19:49:48.368426 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6960a0-8f98-4604-b24d-25da631b4ee1 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:48.369045 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6960a0-8f98-4604-b24d-25da631b4ee1 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:48.369757 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6960a0-8f98-4604-b24d-25da631b4ee1 None admin] PATH_INFO: `/v3/projects` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:48.370728 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f6960a0-8f98-4604-b24d-25da631b4ee1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:49:48.371234 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f6960a0-8f98-4604-b24d-25da631b4ee1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:48.372885 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application [None req-cc56c1fd-36ed-4662-b62c-953465e0c578 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 05 19:49:48.372885 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:49:48.372885 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:49:48.372885 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:49:48.372885 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:49:48.372885 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:49:48.372885 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:49:48.372885 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:49:48.372885 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:49:48.372885 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:49:48.372885 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:49:48.372885 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:49:48.372885 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 19:49:48.372885 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 19:49:48.372885 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 19:49:48.372885 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 19:49:48.374369 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 19:49:48.374369 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 19:49:48.374369 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 19:49:48.374369 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 19:49:48.374369 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:49:48.374369 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:49:48.374369 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:49:48.374369 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:49:48.374369 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:49:48.374369 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:49:48.374369 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:49:48.374369 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application with Lock( Aug 05 19:49:48.374369 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:49:48.374369 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:49:48.374369 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:49:48.374369 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:49:48.375202 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:49:48.375202 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:49:48.375202 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:49:48.375202 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:49:48.375202 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 19:49:48.375202 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 19:49:48.375202 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 19:49:48.375202 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 19:49:48.375202 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 19:49:48.375202 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 19:49:48.375202 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 05 19:49:48.375202 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Aug 05 19:49:48.375202 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 300/608] 10.222.0.27 () {56 vars in 1102 bytes} [Fri Aug 5 19:49:48 2022] GET /identity/v3/roles/service => generated 85 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 05 19:49:48.376377 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954b8cb9-4530-4833-a89d-9ca4caedfd42 None None] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:48.376615 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f6960a0-8f98-4604-b24d-25da631b4ee1 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:48.376722 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954b8cb9-4530-4833-a89d-9ca4caedfd42 None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:48.376949 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954b8cb9-4530-4833-a89d-9ca4caedfd42 None None] PATH_INFO: `/` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:48.382280 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 301/609] 10.222.0.27 () {54 vars in 927 bytes} [Fri Aug 5 19:49:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:49:48.383063 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954b8cb9-4530-4833-a89d-9ca4caedfd42 None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:48.391006 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 309/610] 10.222.0.27 () {56 vars in 1148 bytes} [Fri Aug 5 19:49:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 19:49:48.392581 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03d69c0-3605-4ffb-bbc4-4f3ff603497e None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:48.393092 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03d69c0-3605-4ffb-bbc4-4f3ff603497e None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:48.393568 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03d69c0-3605-4ffb-bbc4-4f3ff603497e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:48.419323 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-c03d69c0-3605-4ffb-bbc4-4f3ff603497e None None] MFA Rules not processed for user `03d1834a64cd4a43829827cb99a72747`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:49:48.421462 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-c03d69c0-3605-4ffb-bbc4-4f3ff603497e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:49:48.449511 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:48.509280 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954b8cb9-4530-4833-a89d-9ca4caedfd42 None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:48.512891 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 310/611] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Aug 5 19:49:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 19:49:48.514343 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03d69c0-3605-4ffb-bbc4-4f3ff603497e None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:48.568991 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a291c0-6323-4e74-83cd-3623b5f91809 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:48.569700 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a291c0-6323-4e74-83cd-3623b5f91809 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:48.569867 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a291c0-6323-4e74-83cd-3623b5f91809 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:48.570191 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a291c0-6323-4e74-83cd-3623b5f91809 None admin] PATH_INFO: `/v3/roles` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:48.570853 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a291c0-6323-4e74-83cd-3623b5f91809 None admin] RBAC: Adding query filter params (name=service) {{(pid=80364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:49:48.571319 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a291c0-6323-4e74-83cd-3623b5f91809 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:48.572789 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a291c0-6323-4e74-83cd-3623b5f91809 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:48.581900 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 302/612] 10.222.0.27 () {56 vars in 1103 bytes} [Fri Aug 5 19:49:48 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:49:48.582774 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a291c0-6323-4e74-83cd-3623b5f91809 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:48.624228 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:48.640511 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03d69c0-3605-4ffb-bbc4-4f3ff603497e None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:48.685018 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cea231b-f0ab-4429-be15-fb72e75167a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:48.686603 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cea231b-f0ab-4429-be15-fb72e75167a9 None admin] REQUEST_METHOD: `PUT` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:48.687106 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cea231b-f0ab-4429-be15-fb72e75167a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:48.687719 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cea231b-f0ab-4429-be15-fb72e75167a9 None admin] PATH_INFO: `/v3/projects/a70a82dcfc574fad90b3f5f00abbba58/users/6e632e7fe71448c29273ac65b5899aca/roles/6e2bc1ba438d4eb69923ef4659c181f3` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:48.699302 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a291c0-6323-4e74-83cd-3623b5f91809 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:48.704379 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cea231b-f0ab-4429-be15-fb72e75167a9 None admin] RBAC: Authorizing `identity:create_grant(project_id=a70a82dcfc574fad90b3f5f00abbba58, user_id=6e632e7fe71448c29273ac65b5899aca, role_id=6e2bc1ba438d4eb69923ef4659c181f3)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:48.705871 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cea231b-f0ab-4429-be15-fb72e75167a9 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:48.708971 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:48.721553 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 311/613] 10.222.0.27 () {58 vars in 1439 bytes} [Fri Aug 5 19:49:48 2022] PUT /identity/v3/projects/a70a82dcfc574fad90b3f5f00abbba58/users/6e632e7fe71448c29273ac65b5899aca/roles/6e2bc1ba438d4eb69923ef4659c181f3 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 19:49:48.731327 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cea231b-f0ab-4429-be15-fb72e75167a9 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:48.742132 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0badd153-fa76-4c35-952c-3b251cd91605 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:48.742792 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0badd153-fa76-4c35-952c-3b251cd91605 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:48.743076 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0badd153-fa76-4c35-952c-3b251cd91605 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:48.743352 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0badd153-fa76-4c35-952c-3b251cd91605 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:48.756973 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0badd153-fa76-4c35-952c-3b251cd91605 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 19:49:48.763517 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application [None req-0badd153-fa76-4c35-952c-3b251cd91605 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 05 19:49:48.763517 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:49:48.763517 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:49:48.763517 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:49:48.763517 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:49:48.763517 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:49:48.763517 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:49:48.763517 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:49:48.763517 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:49:48.763517 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:49:48.763517 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:49:48.763517 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:49:48.763517 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 19:49:48.763517 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 19:49:48.763517 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 19:49:48.763517 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 19:49:48.764612 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 19:49:48.764612 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 19:49:48.764612 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 19:49:48.764612 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 19:49:48.764612 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:49:48.764612 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:49:48.764612 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:49:48.764612 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:49:48.764612 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:49:48.764612 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:49:48.764612 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:49:48.764612 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application with Lock( Aug 05 19:49:48.764612 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:49:48.764612 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:49:48.764612 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:49:48.764612 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:49:48.766016 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:49:48.766016 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:49:48.766016 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:49:48.766016 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:49:48.766016 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 19:49:48.766016 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 19:49:48.766016 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 19:49:48.766016 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 19:49:48.766016 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 19:49:48.766016 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 19:49:48.766016 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 05 19:49:48.766016 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Aug 05 19:49:48.766016 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 303/614] 10.222.0.27 () {56 vars in 1102 bytes} [Fri Aug 5 19:49:48 2022] GET /identity/v3/roles/service => generated 85 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:49:48.775572 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0badd153-fa76-4c35-952c-3b251cd91605 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:48.903749 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0badd153-fa76-4c35-952c-3b251cd91605 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:48.945551 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6af5dee-f396-4d15-aa5f-07e30544efa3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:48.946098 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6af5dee-f396-4d15-aa5f-07e30544efa3 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:48.946697 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6af5dee-f396-4d15-aa5f-07e30544efa3 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:48.946982 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6af5dee-f396-4d15-aa5f-07e30544efa3 None admin] PATH_INFO: `/v3/roles` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:48.947778 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6af5dee-f396-4d15-aa5f-07e30544efa3 None admin] RBAC: Adding query filter params (name=service) {{(pid=80364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:49:48.948083 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6af5dee-f396-4d15-aa5f-07e30544efa3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:48.949514 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6af5dee-f396-4d15-aa5f-07e30544efa3 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:48.960133 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 304/615] 10.222.0.27 () {56 vars in 1103 bytes} [Fri Aug 5 19:49:48 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 19:49:48.966150 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6af5dee-f396-4d15-aa5f-07e30544efa3 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:48.997566 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cea231b-f0ab-4429-be15-fb72e75167a9 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:49.002345 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:49.065228 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e571f5-2406-4593-bbe9-7612b7b20b26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:49.066445 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e571f5-2406-4593-bbe9-7612b7b20b26 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:49.066445 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e571f5-2406-4593-bbe9-7612b7b20b26 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:49.066445 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e571f5-2406-4593-bbe9-7612b7b20b26 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:49.082314 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:49.086008 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e571f5-2406-4593-bbe9-7612b7b20b26 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:49.087816 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e571f5-2406-4593-bbe9-7612b7b20b26 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:49.105441 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6af5dee-f396-4d15-aa5f-07e30544efa3 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:49.113509 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-82e571f5-2406-4593-bbe9-7612b7b20b26 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:49:49.113509 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:49:49.113509 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:49:49.113509 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:49:49.113509 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:49:49.113509 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:49:49.113509 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:49:49.113509 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:49:49.113509 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:49:49.113509 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:49:49.113509 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:49:49.113509 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:49:49.113509 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 19:49:49.113509 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 19:49:49.113509 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 19:49:49.114423 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 19:49:49.114423 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:49:49.114423 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:49:49.114423 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:49:49.114423 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:49:49.114423 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:49:49.114423 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:49:49.114423 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:49:49.114423 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Aug 05 19:49:49.114423 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:49:49.114423 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:49:49.114423 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:49:49.114423 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:49:49.114423 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:49:49.114423 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:49:49.114423 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:49:49.114423 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:49:49.115321 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 19:49:49.115321 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 19:49:49.115321 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:49:49.115321 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Aug 05 19:49:49.115321 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 312/616] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Aug 5 19:49:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 391 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:49:49.121987 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e571f5-2406-4593-bbe9-7612b7b20b26 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:49.169897 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b878c582-5e32-4c45-850a-c74a76f87154 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:49.170586 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b878c582-5e32-4c45-850a-c74a76f87154 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:49.170866 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b878c582-5e32-4c45-850a-c74a76f87154 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:49.171144 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b878c582-5e32-4c45-850a-c74a76f87154 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:49.188376 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b878c582-5e32-4c45-850a-c74a76f87154 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:49.189859 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b878c582-5e32-4c45-850a-c74a76f87154 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:49.214232 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application [None req-b878c582-5e32-4c45-850a-c74a76f87154 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:49:49.214232 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:49:49.214232 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:49:49.214232 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:49:49.214232 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:49:49.214232 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:49:49.214232 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:49:49.214232 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:49:49.214232 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:49:49.214232 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:49:49.214232 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:49:49.214232 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:49:49.214232 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 19:49:49.214232 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 19:49:49.214232 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 19:49:49.215156 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 19:49:49.215156 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:49:49.215156 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:49:49.215156 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:49:49.215156 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:49:49.215156 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:49:49.215156 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:49:49.215156 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:49:49.215156 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application with Lock( Aug 05 19:49:49.215156 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:49:49.215156 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:49:49.215156 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:49:49.215156 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:49:49.215156 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:49:49.215156 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:49:49.215156 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:49:49.215156 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:49:49.216030 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 19:49:49.216030 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 19:49:49.216030 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:49:49.216030 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Aug 05 19:49:49.222033 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 305/617] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Aug 5 19:49:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 05 19:49:49.237908 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b878c582-5e32-4c45-850a-c74a76f87154 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:49.272523 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e571f5-2406-4593-bbe9-7612b7b20b26 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:49.309187 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05692063-7e7d-4d9a-a527-293624e3a52c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:49.309809 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05692063-7e7d-4d9a-a527-293624e3a52c None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:49.310090 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05692063-7e7d-4d9a-a527-293624e3a52c None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:49.310362 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05692063-7e7d-4d9a-a527-293624e3a52c None admin] PATH_INFO: `/v3/domains` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:49.312834 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05692063-7e7d-4d9a-a527-293624e3a52c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:49:49.313159 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05692063-7e7d-4d9a-a527-293624e3a52c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:49.314546 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05692063-7e7d-4d9a-a527-293624e3a52c None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:49.350814 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 313/618] 10.222.0.27 () {56 vars in 1109 bytes} [Fri Aug 5 19:49:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 19:49:49.362637 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05692063-7e7d-4d9a-a527-293624e3a52c None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:49.380853 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:49.407669 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b878c582-5e32-4c45-850a-c74a76f87154 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:49.441462 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:49.448717 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de06244e-9502-4cf9-9ae2-f5076610c261 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:49.449812 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de06244e-9502-4cf9-9ae2-f5076610c261 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:49.450197 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de06244e-9502-4cf9-9ae2-f5076610c261 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:49.450824 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de06244e-9502-4cf9-9ae2-f5076610c261 None admin] PATH_INFO: `/v3/domains` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:49.452194 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de06244e-9502-4cf9-9ae2-f5076610c261 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:49:49.452697 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de06244e-9502-4cf9-9ae2-f5076610c261 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:49.454327 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de06244e-9502-4cf9-9ae2-f5076610c261 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:49.470507 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 306/619] 10.222.0.27 () {56 vars in 1109 bytes} [Fri Aug 5 19:49:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:49:49.472344 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dec6adf-fa9a-44f1-a95a-13229c3e9381 None None] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:49.472765 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dec6adf-fa9a-44f1-a95a-13229c3e9381 None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:49.473143 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dec6adf-fa9a-44f1-a95a-13229c3e9381 None None] PATH_INFO: `/` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:49.476566 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 307/620] 10.222.0.27 () {54 vars in 927 bytes} [Fri Aug 5 19:49:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:49:49.477508 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dec6adf-fa9a-44f1-a95a-13229c3e9381 None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:49.502738 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05692063-7e7d-4d9a-a527-293624e3a52c None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:49.539153 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cba8f8f-c46e-4063-baf7-b7f358f78d0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:49.540666 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cba8f8f-c46e-4063-baf7-b7f358f78d0a None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:49.541122 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cba8f8f-c46e-4063-baf7-b7f358f78d0a None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:49.541549 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cba8f8f-c46e-4063-baf7-b7f358f78d0a None admin] PATH_INFO: `/v3/users/swift` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:49.549661 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cba8f8f-c46e-4063-baf7-b7f358f78d0a None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:49.551300 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cba8f8f-c46e-4063-baf7-b7f358f78d0a None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:49.565741 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-9cba8f8f-c46e-4063-baf7-b7f358f78d0a None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 05 19:49:49.565741 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:49:49.565741 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:49:49.565741 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:49:49.565741 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:49:49.565741 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:49:49.565741 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:49:49.565741 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:49:49.565741 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:49:49.565741 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:49:49.565741 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:49:49.565741 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:49:49.565741 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 05 19:49:49.565741 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 05 19:49:49.565741 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 05 19:49:49.565741 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 05 19:49:49.566734 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:49:49.566734 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:49:49.566734 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 05 19:49:49.566734 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 19:49:49.566734 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 05 19:49:49.566734 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 19:49:49.566734 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:49:49.566734 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:49:49.566734 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:49:49.566734 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:49:49.566734 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:49:49.566734 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Aug 05 19:49:49.566734 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:49:49.566734 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:49:49.566734 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:49:49.566734 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:49:49.566734 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:49:49.566734 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:49:49.567665 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:49:49.567665 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:49:49.567665 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 05 19:49:49.567665 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 05 19:49:49.567665 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 05 19:49:49.567665 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 05 19:49:49.567665 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 05 19:49:49.567665 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 05 19:49:49.567665 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 05 19:49:49.567665 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Aug 05 19:49:49.570268 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 314/621] 10.222.0.27 () {56 vars in 1096 bytes} [Fri Aug 5 19:49:49 2022] GET /identity/v3/users/swift => generated 83 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:49:49.571699 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bc2ef1-555f-42ad-92c3-4d32d58a15b5 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:49.572220 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bc2ef1-555f-42ad-92c3-4d32d58a15b5 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:49.572689 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bc2ef1-555f-42ad-92c3-4d32d58a15b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:49.591916 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-d7bc2ef1-555f-42ad-92c3-4d32d58a15b5 None None] MFA Rules not processed for user `03d1834a64cd4a43829827cb99a72747`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:49:49.593508 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-d7bc2ef1-555f-42ad-92c3-4d32d58a15b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:49:49.598840 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dec6adf-fa9a-44f1-a95a-13229c3e9381 None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:49.636249 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c189a1b-5c52-455b-8db0-7f7a458f21c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:49.636936 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c189a1b-5c52-455b-8db0-7f7a458f21c7 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:49.637214 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c189a1b-5c52-455b-8db0-7f7a458f21c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:49.637490 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c189a1b-5c52-455b-8db0-7f7a458f21c7 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:49.645990 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c189a1b-5c52-455b-8db0-7f7a458f21c7 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:49.647454 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c189a1b-5c52-455b-8db0-7f7a458f21c7 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:49.650514 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:49.661686 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application [None req-9c189a1b-5c52-455b-8db0-7f7a458f21c7 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 05 19:49:49.661686 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:49:49.661686 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:49:49.661686 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:49:49.661686 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:49:49.661686 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:49:49.661686 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:49:49.661686 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:49:49.661686 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:49:49.661686 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:49:49.661686 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:49:49.661686 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:49:49.661686 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 05 19:49:49.661686 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 05 19:49:49.661686 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 05 19:49:49.661686 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 05 19:49:49.662711 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:49:49.662711 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:49:49.662711 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 05 19:49:49.662711 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 19:49:49.662711 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 05 19:49:49.662711 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 19:49:49.662711 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:49:49.662711 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:49:49.662711 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:49:49.662711 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:49:49.662711 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:49:49.662711 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application with Lock( Aug 05 19:49:49.662711 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:49:49.662711 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:49:49.662711 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:49:49.662711 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:49:49.662711 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:49:49.662711 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:49:49.663681 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:49:49.663681 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:49:49.663681 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 05 19:49:49.663681 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 05 19:49:49.663681 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 05 19:49:49.663681 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 05 19:49:49.663681 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 05 19:49:49.663681 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 05 19:49:49.663681 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 05 19:49:49.663681 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Aug 05 19:49:49.663681 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 308/622] 10.222.0.27 () {56 vars in 1093 bytes} [Fri Aug 5 19:49:49 2022] GET /identity/v3/users/nova => generated 82 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:49:49.664285 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c189a1b-5c52-455b-8db0-7f7a458f21c7 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:49.673016 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 315/623] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Aug 5 19:49:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 19:49:49.674186 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7bc2ef1-555f-42ad-92c3-4d32d58a15b5 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:49.718477 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:49.782616 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7bc2ef1-555f-42ad-92c3-4d32d58a15b5 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:49.802601 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c189a1b-5c52-455b-8db0-7f7a458f21c7 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:49.819833 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c65a7c2-720f-44fb-9699-9c64817b9fbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:49.820839 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c65a7c2-720f-44fb-9699-9c64817b9fbc None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:49.821338 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c65a7c2-720f-44fb-9699-9c64817b9fbc None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:49.821802 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c65a7c2-720f-44fb-9699-9c64817b9fbc None admin] PATH_INFO: `/v3/users` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:49.822846 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c65a7c2-720f-44fb-9699-9c64817b9fbc None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:49:49.823374 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c65a7c2-720f-44fb-9699-9c64817b9fbc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:49.825045 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c65a7c2-720f-44fb-9699-9c64817b9fbc None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:49.839025 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c816ae-19a7-44be-9c6d-ea41cf24bfae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:49.839647 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c816ae-19a7-44be-9c6d-ea41cf24bfae None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:49.839924 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c816ae-19a7-44be-9c6d-ea41cf24bfae None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:49.840196 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c816ae-19a7-44be-9c6d-ea41cf24bfae None admin] PATH_INFO: `/v3/users` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:49.841315 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c816ae-19a7-44be-9c6d-ea41cf24bfae None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:49:49.841315 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c816ae-19a7-44be-9c6d-ea41cf24bfae None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:49.842783 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c816ae-19a7-44be-9c6d-ea41cf24bfae None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:49.845501 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 316/624] 10.222.0.27 () {56 vars in 1133 bytes} [Fri Aug 5 19:49:49 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:49:49.846620 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c65a7c2-720f-44fb-9699-9c64817b9fbc None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:49.862188 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 309/625] 10.222.0.27 () {56 vars in 1135 bytes} [Fri Aug 5 19:49:49 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:49:49.863034 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c816ae-19a7-44be-9c6d-ea41cf24bfae None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:49.925772 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:49.949409 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:49.954003 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c65a7c2-720f-44fb-9699-9c64817b9fbc None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:49.973578 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c816ae-19a7-44be-9c6d-ea41cf24bfae None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:49.991324 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61a4d34-e471-4ad3-b9a8-854d312be92f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:49.991971 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61a4d34-e471-4ad3-b9a8-854d312be92f None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:49.992254 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61a4d34-e471-4ad3-b9a8-854d312be92f None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:49.992549 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61a4d34-e471-4ad3-b9a8-854d312be92f None admin] PATH_INFO: `/v3/roles/service` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:50.003447 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a61a4d34-e471-4ad3-b9a8-854d312be92f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 19:49:50.008353 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-a61a4d34-e471-4ad3-b9a8-854d312be92f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 05 19:49:50.008353 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:49:50.008353 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:49:50.008353 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:49:50.008353 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:49:50.008353 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:49:50.008353 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:49:50.008353 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:49:50.008353 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:49:50.008353 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:49:50.008353 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:49:50.008353 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:49:50.008353 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 19:49:50.008353 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 19:49:50.008353 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 19:49:50.008353 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 19:49:50.009407 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 19:49:50.009407 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 19:49:50.009407 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 19:49:50.009407 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 19:49:50.009407 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:49:50.009407 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:49:50.009407 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:49:50.009407 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:49:50.009407 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:49:50.009407 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:49:50.009407 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:49:50.009407 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Aug 05 19:49:50.009407 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:49:50.009407 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:49:50.009407 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:49:50.009407 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:49:50.010325 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:49:50.010325 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:49:50.010325 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:49:50.010325 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:49:50.010325 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 19:49:50.010325 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 19:49:50.010325 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 19:49:50.010325 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 19:49:50.010325 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 19:49:50.010325 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 19:49:50.010325 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 05 19:49:50.010325 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Aug 05 19:49:50.010325 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 317/626] 10.222.0.27 () {56 vars in 1102 bytes} [Fri Aug 5 19:49:49 2022] GET /identity/v3/roles/service => generated 85 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:49:50.011024 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61a4d34-e471-4ad3-b9a8-854d312be92f None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:50.017484 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfaca2ba-ae21-4521-b6ec-271a356108e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:50.018436 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfaca2ba-ae21-4521-b6ec-271a356108e7 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:50.018793 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfaca2ba-ae21-4521-b6ec-271a356108e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:50.019071 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfaca2ba-ae21-4521-b6ec-271a356108e7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:50.094264 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfaca2ba-ae21-4521-b6ec-271a356108e7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:50.095648 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfaca2ba-ae21-4521-b6ec-271a356108e7 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:50.153566 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61a4d34-e471-4ad3-b9a8-854d312be92f None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:50.156012 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:50.198383 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f49744d-7540-429d-aafc-1c49ecc96aa8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:50.199026 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f49744d-7540-429d-aafc-1c49ecc96aa8 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:50.199305 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f49744d-7540-429d-aafc-1c49ecc96aa8 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:50.199575 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f49744d-7540-429d-aafc-1c49ecc96aa8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:50.213936 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f49744d-7540-429d-aafc-1c49ecc96aa8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:50.213936 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f49744d-7540-429d-aafc-1c49ecc96aa8 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:50.214197 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 310/627] 10.222.0.27 () {58 vars in 1282 bytes} [Fri Aug 5 19:49:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 05 19:49:50.214197 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52894d41-74bc-4957-a70f-6432defe5425 None None] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:50.214865 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52894d41-74bc-4957-a70f-6432defe5425 None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:50.214865 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52894d41-74bc-4957-a70f-6432defe5425 None None] PATH_INFO: `/` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:50.216448 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 311/628] 10.222.0.27 () {54 vars in 927 bytes} [Fri Aug 5 19:49:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:49:50.217533 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52894d41-74bc-4957-a70f-6432defe5425 None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:50.235089 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-1f49744d-7540-429d-aafc-1c49ecc96aa8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:49:50.235089 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:49:50.235089 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:49:50.235089 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:49:50.235089 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:49:50.235089 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:49:50.235089 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:49:50.235089 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:49:50.235089 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:49:50.235089 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:49:50.235089 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:49:50.235089 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:49:50.235089 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 19:49:50.235089 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 19:49:50.235089 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 19:49:50.236227 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 19:49:50.236227 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:49:50.236227 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:49:50.236227 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:49:50.236227 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:49:50.236227 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:49:50.236227 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:49:50.236227 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:49:50.236227 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Aug 05 19:49:50.236227 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:49:50.236227 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:49:50.236227 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:49:50.236227 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:49:50.236227 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:49:50.236227 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:49:50.236227 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:49:50.236227 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:49:50.237320 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 19:49:50.237320 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 19:49:50.237320 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:49:50.237320 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Aug 05 19:49:50.237881 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 318/629] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Aug 5 19:49:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:49:50.238721 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f49744d-7540-429d-aafc-1c49ecc96aa8 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:50.253446 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:50.333256 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52894d41-74bc-4957-a70f-6432defe5425 None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:50.344895 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f49744d-7540-429d-aafc-1c49ecc96aa8 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:50.378553 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49629e2f-3afe-45f0-97f1-8cb95f3b9539 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:50.379168 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49629e2f-3afe-45f0-97f1-8cb95f3b9539 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:50.379449 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49629e2f-3afe-45f0-97f1-8cb95f3b9539 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:50.379728 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49629e2f-3afe-45f0-97f1-8cb95f3b9539 None admin] PATH_INFO: `/v3/roles` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:50.380441 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49629e2f-3afe-45f0-97f1-8cb95f3b9539 None admin] RBAC: Adding query filter params (name=service) {{(pid=80364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:49:50.380771 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49629e2f-3afe-45f0-97f1-8cb95f3b9539 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:50.382204 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49629e2f-3afe-45f0-97f1-8cb95f3b9539 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:50.385701 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db88440-e479-4015-817b-6eda56222bd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:50.386339 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db88440-e479-4015-817b-6eda56222bd6 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:50.386616 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db88440-e479-4015-817b-6eda56222bd6 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:50.386885 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db88440-e479-4015-817b-6eda56222bd6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:50.394909 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 312/630] 10.222.0.27 () {56 vars in 1103 bytes} [Fri Aug 5 19:49:50 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:49:50.396033 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8394e414-324d-442f-b7c4-0fb46d293db7 None None] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:50.396318 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8394e414-324d-442f-b7c4-0fb46d293db7 None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:50.396611 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8394e414-324d-442f-b7c4-0fb46d293db7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:50.398886 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db88440-e479-4015-817b-6eda56222bd6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:50.400696 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db88440-e479-4015-817b-6eda56222bd6 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:50.415460 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.auth.core [None req-8394e414-324d-442f-b7c4-0fb46d293db7 None None] MFA Rules not processed for user `03d1834a64cd4a43829827cb99a72747`. Rule list: `[]` (Enabled: `True`). {{(pid=80364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:49:50.415961 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-0db88440-e479-4015-817b-6eda56222bd6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:49:50.415961 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:49:50.415961 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:49:50.415961 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:49:50.415961 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:49:50.415961 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:49:50.415961 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:49:50.415961 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:49:50.415961 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:49:50.415961 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:49:50.415961 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:49:50.415961 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:49:50.415961 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 19:49:50.415961 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 19:49:50.415961 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 19:49:50.417021 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 19:49:50.417021 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:49:50.417021 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:49:50.417021 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:49:50.417021 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:49:50.417021 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:49:50.417021 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:49:50.417021 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:49:50.417021 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Aug 05 19:49:50.417021 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:49:50.417021 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:49:50.417021 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:49:50.417021 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:49:50.417021 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:49:50.417021 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:49:50.417021 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:49:50.417021 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:49:50.418101 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 19:49:50.418101 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 19:49:50.418101 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:49:50.418101 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Aug 05 19:49:50.418368 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-8394e414-324d-442f-b7c4-0fb46d293db7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:49:50.419361 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 319/631] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Aug 5 19:49:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:49:50.420782 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db88440-e479-4015-817b-6eda56222bd6 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:50.437746 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:50.499985 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:50.506847 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 313/632] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Aug 5 19:49:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 19:49:50.507054 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8394e414-324d-442f-b7c4-0fb46d293db7 None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:50.551634 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db88440-e479-4015-817b-6eda56222bd6 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:50.589846 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f47368-a991-4340-a5d2-db1f7c55f9d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:50.590465 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f47368-a991-4340-a5d2-db1f7c55f9d6 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:50.590745 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f47368-a991-4340-a5d2-db1f7c55f9d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:50.591019 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f47368-a991-4340-a5d2-db1f7c55f9d6 None admin] PATH_INFO: `/v3/domains` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:50.591715 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f47368-a991-4340-a5d2-db1f7c55f9d6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:49:50.592723 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f47368-a991-4340-a5d2-db1f7c55f9d6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:50.594066 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f47368-a991-4340-a5d2-db1f7c55f9d6 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:50.607319 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 320/633] 10.222.0.27 () {56 vars in 1109 bytes} [Fri Aug 5 19:49:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:49:50.608454 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f47368-a991-4340-a5d2-db1f7c55f9d6 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:50.624306 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8394e414-324d-442f-b7c4-0fb46d293db7 None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:50.662903 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ff044c-4bcf-47c4-b1ed-67f17e47505c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:50.663547 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ff044c-4bcf-47c4-b1ed-67f17e47505c None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:50.663827 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ff044c-4bcf-47c4-b1ed-67f17e47505c None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:50.664105 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ff044c-4bcf-47c4-b1ed-67f17e47505c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:50.675317 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ff044c-4bcf-47c4-b1ed-67f17e47505c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:50.676997 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ff044c-4bcf-47c4-b1ed-67f17e47505c None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:50.691898 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application [None req-a2ff044c-4bcf-47c4-b1ed-67f17e47505c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:49:50.691898 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:49:50.691898 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:49:50.691898 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:49:50.691898 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:49:50.691898 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:49:50.691898 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:49:50.691898 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:49:50.691898 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:49:50.691898 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:49:50.691898 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:49:50.691898 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:49:50.691898 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 19:49:50.691898 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 19:49:50.691898 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 19:49:50.692981 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 19:49:50.692981 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:49:50.692981 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:49:50.692981 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:49:50.692981 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:49:50.692981 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:49:50.692981 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:49:50.692981 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:49:50.692981 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application with Lock( Aug 05 19:49:50.692981 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:49:50.692981 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:49:50.692981 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:49:50.692981 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:49:50.692981 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:49:50.692981 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:49:50.692981 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:49:50.692981 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:49:50.694050 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 19:49:50.694050 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 19:49:50.694050 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:49:50.694050 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Aug 05 19:49:50.700135 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 314/634] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Aug 5 19:49:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:49:50.701235 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ff044c-4bcf-47c4-b1ed-67f17e47505c None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:50.706111 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:50.714312 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f47368-a991-4340-a5d2-db1f7c55f9d6 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:50.737943 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:50.750357 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c1e87e-0bd1-409b-a8e6-fad02a8a2af2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:50.750967 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c1e87e-0bd1-409b-a8e6-fad02a8a2af2 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:50.751239 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c1e87e-0bd1-409b-a8e6-fad02a8a2af2 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:50.751509 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c1e87e-0bd1-409b-a8e6-fad02a8a2af2 None admin] PATH_INFO: `/v3/domains` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:50.752204 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c1e87e-0bd1-409b-a8e6-fad02a8a2af2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:49:50.752522 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c1e87e-0bd1-409b-a8e6-fad02a8a2af2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:50.753883 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c1e87e-0bd1-409b-a8e6-fad02a8a2af2 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:50.765876 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 321/635] 10.222.0.27 () {56 vars in 1109 bytes} [Fri Aug 5 19:49:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:49:50.766563 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c1e87e-0bd1-409b-a8e6-fad02a8a2af2 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:50.817207 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ff044c-4bcf-47c4-b1ed-67f17e47505c None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:50.851248 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02bb51eb-d093-44d4-b40a-9bc5ea155278 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:50.851975 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bb51eb-d093-44d4-b40a-9bc5ea155278 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:50.852323 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bb51eb-d093-44d4-b40a-9bc5ea155278 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:50.852690 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bb51eb-d093-44d4-b40a-9bc5ea155278 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:50.864464 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02bb51eb-d093-44d4-b40a-9bc5ea155278 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 19:49:50.869855 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application [None req-02bb51eb-d093-44d4-b40a-9bc5ea155278 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 05 19:49:50.869855 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:49:50.869855 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:49:50.869855 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:49:50.869855 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:49:50.869855 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:49:50.869855 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:49:50.869855 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:49:50.869855 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:49:50.869855 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:49:50.869855 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:49:50.869855 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:49:50.869855 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 19:49:50.869855 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 19:49:50.869855 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 19:49:50.869855 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 19:49:50.870995 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 19:49:50.870995 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 19:49:50.870995 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 19:49:50.870995 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 19:49:50.870995 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:49:50.870995 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:49:50.870995 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:49:50.870995 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:49:50.870995 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:49:50.870995 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:49:50.870995 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:49:50.870995 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application with Lock( Aug 05 19:49:50.870995 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:49:50.870995 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:49:50.870995 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:49:50.870995 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:49:50.872017 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:49:50.872017 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:49:50.872017 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:49:50.872017 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:49:50.872017 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 19:49:50.872017 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 19:49:50.872017 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 19:49:50.872017 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 19:49:50.872017 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 19:49:50.872017 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 19:49:50.872017 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 05 19:49:50.872017 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Aug 05 19:49:50.873579 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c1e87e-0bd1-409b-a8e6-fad02a8a2af2 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:50.877981 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 315/636] 10.222.0.27 () {56 vars in 1102 bytes} [Fri Aug 5 19:49:50 2022] GET /identity/v3/roles/service => generated 85 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:49:50.878778 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02bb51eb-d093-44d4-b40a-9bc5ea155278 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:50.910520 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a9d39b-5a22-426e-8728-d41bbe1c1a9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:50.911153 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a9d39b-5a22-426e-8728-d41bbe1c1a9b None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:50.911427 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a9d39b-5a22-426e-8728-d41bbe1c1a9b None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:50.911696 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a9d39b-5a22-426e-8728-d41bbe1c1a9b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:50.920074 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a9d39b-5a22-426e-8728-d41bbe1c1a9b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:50.921677 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a9d39b-5a22-426e-8728-d41bbe1c1a9b None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:50.933003 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-69a9d39b-5a22-426e-8728-d41bbe1c1a9b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 19:49:50.933003 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:49:50.933003 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:49:50.933003 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:49:50.933003 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:49:50.933003 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:49:50.933003 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:49:50.933003 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:49:50.933003 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:49:50.933003 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:49:50.933003 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:49:50.933003 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:49:50.933003 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 05 19:49:50.933003 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 05 19:49:50.933003 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 05 19:49:50.934163 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 05 19:49:50.934163 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:49:50.934163 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:49:50.934163 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:49:50.934163 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:49:50.934163 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:49:50.934163 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:49:50.934163 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:49:50.934163 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Aug 05 19:49:50.934163 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:49:50.934163 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:49:50.934163 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:49:50.934163 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:49:50.934163 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:49:50.934163 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:49:50.934163 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:49:50.934163 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:49:50.935244 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 05 19:49:50.935244 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 05 19:49:50.935244 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 05 19:49:50.935244 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 05 19:49:50.935244 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 05 19:49:50.935244 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 05 19:49:50.935244 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 19:49:50.935244 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Aug 05 19:49:50.939455 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 322/637] 10.222.0.27 () {56 vars in 1111 bytes} [Fri Aug 5 19:49:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:49:50.940053 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a9d39b-5a22-426e-8728-d41bbe1c1a9b None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:50.943294 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:50.983591 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02bb51eb-d093-44d4-b40a-9bc5ea155278 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:50.986346 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:51.020040 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a03aef-d76f-43ed-8ff9-61df574c434d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:51.020683 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a03aef-d76f-43ed-8ff9-61df574c434d None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:51.020963 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a03aef-d76f-43ed-8ff9-61df574c434d None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:51.021231 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a03aef-d76f-43ed-8ff9-61df574c434d None admin] PATH_INFO: `/v3/domains` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:51.022032 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a03aef-d76f-43ed-8ff9-61df574c434d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:49:51.022340 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a03aef-d76f-43ed-8ff9-61df574c434d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:51.023701 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a03aef-d76f-43ed-8ff9-61df574c434d None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:51.036058 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 316/638] 10.222.0.27 () {56 vars in 1109 bytes} [Fri Aug 5 19:49:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:49:51.036871 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a03aef-d76f-43ed-8ff9-61df574c434d None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:51.051536 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a9d39b-5a22-426e-8728-d41bbe1c1a9b None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:51.084731 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1736c53d-d488-4589-aa26-6ebaa77a8049 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:51.085364 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1736c53d-d488-4589-aa26-6ebaa77a8049 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:51.085638 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1736c53d-d488-4589-aa26-6ebaa77a8049 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:51.085904 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1736c53d-d488-4589-aa26-6ebaa77a8049 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:51.092911 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1736c53d-d488-4589-aa26-6ebaa77a8049 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:51.094333 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1736c53d-d488-4589-aa26-6ebaa77a8049 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:51.111459 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-1736c53d-d488-4589-aa26-6ebaa77a8049 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 19:49:51.111459 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:49:51.111459 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:49:51.111459 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:49:51.111459 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:49:51.111459 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:49:51.111459 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:49:51.111459 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:49:51.111459 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:49:51.111459 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:49:51.111459 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:49:51.111459 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:49:51.111459 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 05 19:49:51.111459 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 05 19:49:51.111459 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 05 19:49:51.112643 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 05 19:49:51.112643 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:49:51.112643 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:49:51.112643 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:49:51.112643 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:49:51.112643 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:49:51.112643 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:49:51.112643 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:49:51.112643 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Aug 05 19:49:51.112643 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:49:51.112643 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:49:51.112643 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:49:51.112643 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:49:51.112643 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:49:51.112643 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:49:51.112643 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:49:51.112643 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:49:51.113781 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 05 19:49:51.113781 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 05 19:49:51.113781 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 05 19:49:51.113781 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 05 19:49:51.113781 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 05 19:49:51.113781 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 05 19:49:51.113781 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 19:49:51.113781 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Aug 05 19:49:51.113781 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 323/639] 10.222.0.27 () {56 vars in 1111 bytes} [Fri Aug 5 19:49:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:49:51.113781 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1736c53d-d488-4589-aa26-6ebaa77a8049 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:51.138698 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a03aef-d76f-43ed-8ff9-61df574c434d None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:51.164091 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:51.176822 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b6b76e-ddab-452b-8b8d-a610cd2e80b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:51.177439 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b6b76e-ddab-452b-8b8d-a610cd2e80b3 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:51.177723 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b6b76e-ddab-452b-8b8d-a610cd2e80b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:51.177996 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b6b76e-ddab-452b-8b8d-a610cd2e80b3 None admin] PATH_INFO: `/v3/roles` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:51.178708 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b6b76e-ddab-452b-8b8d-a610cd2e80b3 None admin] RBAC: Adding query filter params (name=service) {{(pid=80364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:49:51.179011 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b6b76e-ddab-452b-8b8d-a610cd2e80b3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:51.180381 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b6b76e-ddab-452b-8b8d-a610cd2e80b3 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:51.188234 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 317/640] 10.222.0.27 () {56 vars in 1103 bytes} [Fri Aug 5 19:49:51 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:49:51.189488 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753aaa89-ee46-446e-b965-67622ed2f6a9 None None] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:51.189779 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753aaa89-ee46-446e-b965-67622ed2f6a9 None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:51.190051 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753aaa89-ee46-446e-b965-67622ed2f6a9 None None] PATH_INFO: `/` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:51.191252 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 318/641] 10.222.0.27 () {54 vars in 927 bytes} [Fri Aug 5 19:49:51 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:49:51.191937 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753aaa89-ee46-446e-b965-67622ed2f6a9 None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:51.217690 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1736c53d-d488-4589-aa26-6ebaa77a8049 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:51.219174 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:51.250973 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313a02ac-c6b0-45e2-8d0a-d9e8a96b5903 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:51.251676 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313a02ac-c6b0-45e2-8d0a-d9e8a96b5903 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:51.251961 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313a02ac-c6b0-45e2-8d0a-d9e8a96b5903 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:51.252233 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313a02ac-c6b0-45e2-8d0a-d9e8a96b5903 None admin] PATH_INFO: `/v3/projects` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:51.253010 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313a02ac-c6b0-45e2-8d0a-d9e8a96b5903 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:49:51.253315 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313a02ac-c6b0-45e2-8d0a-d9e8a96b5903 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:51.254738 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313a02ac-c6b0-45e2-8d0a-d9e8a96b5903 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:51.266931 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 324/642] 10.222.0.27 () {56 vars in 1148 bytes} [Fri Aug 5 19:49:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:49:51.267724 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313a02ac-c6b0-45e2-8d0a-d9e8a96b5903 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:51.293046 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753aaa89-ee46-446e-b965-67622ed2f6a9 None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:51.325657 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aeacb94-b386-4f05-b2eb-244dab5d3d0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:51.326313 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aeacb94-b386-4f05-b2eb-244dab5d3d0d None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:51.327092 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aeacb94-b386-4f05-b2eb-244dab5d3d0d None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:51.327361 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aeacb94-b386-4f05-b2eb-244dab5d3d0d None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:51.334564 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aeacb94-b386-4f05-b2eb-244dab5d3d0d None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:51.335980 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aeacb94-b386-4f05-b2eb-244dab5d3d0d None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:51.347659 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application [None req-0aeacb94-b386-4f05-b2eb-244dab5d3d0d None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 05 19:49:51.347659 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:49:51.347659 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:49:51.347659 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:49:51.347659 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:49:51.347659 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:49:51.347659 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:49:51.347659 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:49:51.347659 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:49:51.347659 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:49:51.347659 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:49:51.347659 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:49:51.347659 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 05 19:49:51.347659 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 05 19:49:51.347659 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 05 19:49:51.347659 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 05 19:49:51.348814 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:49:51.348814 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:49:51.348814 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 05 19:49:51.348814 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 19:49:51.348814 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 05 19:49:51.348814 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 19:49:51.348814 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:49:51.348814 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:49:51.348814 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:49:51.348814 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:49:51.348814 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:49:51.348814 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application with Lock( Aug 05 19:49:51.348814 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:49:51.348814 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:49:51.348814 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:49:51.348814 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:49:51.348814 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:49:51.348814 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:49:51.350126 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:49:51.350126 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:49:51.350126 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 05 19:49:51.350126 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 05 19:49:51.350126 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 05 19:49:51.350126 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 05 19:49:51.350126 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 05 19:49:51.350126 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 05 19:49:51.350126 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 05 19:49:51.350126 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Aug 05 19:49:51.350126 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 319/643] 10.222.0.27 () {56 vars in 1099 bytes} [Fri Aug 5 19:49:51 2022] GET /identity/v3/users/cinder => generated 84 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:49:51.351842 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aeacb94-b386-4f05-b2eb-244dab5d3d0d None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:51.373835 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313a02ac-c6b0-45e2-8d0a-d9e8a96b5903 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:51.397155 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:51.408505 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d467be6c-d037-46a1-bb05-269208251e8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:51.409139 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d467be6c-d037-46a1-bb05-269208251e8e None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:51.409413 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d467be6c-d037-46a1-bb05-269208251e8e None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:51.409680 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d467be6c-d037-46a1-bb05-269208251e8e None admin] PATH_INFO: `/v3/projects` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:51.410421 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d467be6c-d037-46a1-bb05-269208251e8e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:49:51.410720 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d467be6c-d037-46a1-bb05-269208251e8e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:51.412127 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d467be6c-d037-46a1-bb05-269208251e8e None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:51.424494 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 325/644] 10.222.0.27 () {56 vars in 1148 bytes} [Fri Aug 5 19:49:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:49:51.425732 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33cea7f-ed42-4682-adc5-e9780585c98f None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:51.426024 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33cea7f-ed42-4682-adc5-e9780585c98f None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:51.426298 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33cea7f-ed42-4682-adc5-e9780585c98f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:51.444526 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-a33cea7f-ed42-4682-adc5-e9780585c98f None None] MFA Rules not processed for user `03d1834a64cd4a43829827cb99a72747`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:49:51.445725 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-a33cea7f-ed42-4682-adc5-e9780585c98f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:49:51.455583 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aeacb94-b386-4f05-b2eb-244dab5d3d0d None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:51.464943 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:51.490320 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5279cc12-e740-485c-83ba-13c2cada0caa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:51.491054 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5279cc12-e740-485c-83ba-13c2cada0caa None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:51.491335 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5279cc12-e740-485c-83ba-13c2cada0caa None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:51.491612 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5279cc12-e740-485c-83ba-13c2cada0caa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:51.502461 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5279cc12-e740-485c-83ba-13c2cada0caa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:51.503950 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5279cc12-e740-485c-83ba-13c2cada0caa None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:51.514423 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 326/645] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Aug 5 19:49:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 19:49:51.515231 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33cea7f-ed42-4682-adc5-e9780585c98f None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:51.518532 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application [None req-5279cc12-e740-485c-83ba-13c2cada0caa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:49:51.518532 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:49:51.518532 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:49:51.518532 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:49:51.518532 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:49:51.518532 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:49:51.518532 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:49:51.518532 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:49:51.518532 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:49:51.518532 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:49:51.518532 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:49:51.518532 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:49:51.518532 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 19:49:51.518532 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 19:49:51.518532 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 19:49:51.519590 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 19:49:51.519590 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:49:51.519590 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:49:51.519590 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:49:51.519590 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:49:51.519590 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:49:51.519590 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:49:51.519590 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:49:51.519590 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application with Lock( Aug 05 19:49:51.519590 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:49:51.519590 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:49:51.519590 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:49:51.519590 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:49:51.519590 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:49:51.519590 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:49:51.519590 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:49:51.519590 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:49:51.520752 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 19:49:51.520752 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 19:49:51.520752 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:49:51.520752 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Aug 05 19:49:51.525848 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 320/646] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Aug 5 19:49:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:49:51.525848 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5279cc12-e740-485c-83ba-13c2cada0caa None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:51.612655 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33cea7f-ed42-4682-adc5-e9780585c98f None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:51.624808 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:51.631767 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5279cc12-e740-485c-83ba-13c2cada0caa None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:51.646497 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7228c4f2-6cb0-438a-af69-a7f2f34b60a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:51.647212 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7228c4f2-6cb0-438a-af69-a7f2f34b60a4 None admin] REQUEST_METHOD: `PUT` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:51.647449 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7228c4f2-6cb0-438a-af69-a7f2f34b60a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:51.647725 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7228c4f2-6cb0-438a-af69-a7f2f34b60a4 None admin] PATH_INFO: `/v3/projects/a70a82dcfc574fad90b3f5f00abbba58/users/6533931da2d34447881008b43372b0d5/roles/6e2bc1ba438d4eb69923ef4659c181f3` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:51.662638 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7228c4f2-6cb0-438a-af69-a7f2f34b60a4 None admin] RBAC: Authorizing `identity:create_grant(project_id=a70a82dcfc574fad90b3f5f00abbba58, user_id=6533931da2d34447881008b43372b0d5, role_id=6e2bc1ba438d4eb69923ef4659c181f3)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:51.664085 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7228c4f2-6cb0-438a-af69-a7f2f34b60a4 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:51.665629 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b1273c-1cf5-4581-adf6-c177e43b304a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:51.666250 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b1273c-1cf5-4581-adf6-c177e43b304a None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:51.666529 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b1273c-1cf5-4581-adf6-c177e43b304a None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:51.666802 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b1273c-1cf5-4581-adf6-c177e43b304a None admin] PATH_INFO: `/v3/users` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:51.667608 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b1273c-1cf5-4581-adf6-c177e43b304a None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:49:51.667912 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b1273c-1cf5-4581-adf6-c177e43b304a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:51.670314 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b1273c-1cf5-4581-adf6-c177e43b304a None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:51.676396 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 327/647] 10.222.0.27 () {58 vars in 1439 bytes} [Fri Aug 5 19:49:51 2022] PUT /identity/v3/projects/a70a82dcfc574fad90b3f5f00abbba58/users/6533931da2d34447881008b43372b0d5/roles/6e2bc1ba438d4eb69923ef4659c181f3 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 19:49:51.677289 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7228c4f2-6cb0-438a-af69-a7f2f34b60a4 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:51.679732 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:51.690123 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 321/648] 10.222.0.27 () {56 vars in 1137 bytes} [Fri Aug 5 19:49:51 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:49:51.690956 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b1273c-1cf5-4581-adf6-c177e43b304a None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:51.806853 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7228c4f2-6cb0-438a-af69-a7f2f34b60a4 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:51.807302 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b1273c-1cf5-4581-adf6-c177e43b304a None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:51.848235 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d0433f-9e0c-468c-b11f-8e4649acd1ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:51.848883 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d0433f-9e0c-468c-b11f-8e4649acd1ae None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:51.849170 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d0433f-9e0c-468c-b11f-8e4649acd1ae None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:51.849444 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d0433f-9e0c-468c-b11f-8e4649acd1ae None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:51.850236 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d0433f-9e0c-468c-b11f-8e4649acd1ae None admin] RBAC: Adding query filter params (role.id=6e2bc1ba438d4eb69923ef4659c181f3, scope.project.id=a70a82dcfc574fad90b3f5f00abbba58, user.id=11001a2ad96f4ec8a35aab2c64376875) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:49:51.850541 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d0433f-9e0c-468c-b11f-8e4649acd1ae None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:51.850948 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37db5f20-a9e7-46bf-b8b4-7340c85c9755 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:51.851678 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37db5f20-a9e7-46bf-b8b4-7340c85c9755 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:51.851982 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d0433f-9e0c-468c-b11f-8e4649acd1ae None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:51.852173 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37db5f20-a9e7-46bf-b8b4-7340c85c9755 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:51.852708 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37db5f20-a9e7-46bf-b8b4-7340c85c9755 None admin] PATH_INFO: `/v3/domains` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:51.853726 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37db5f20-a9e7-46bf-b8b4-7340c85c9755 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:49:51.854153 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37db5f20-a9e7-46bf-b8b4-7340c85c9755 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:51.855597 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37db5f20-a9e7-46bf-b8b4-7340c85c9755 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:51.859329 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 328/649] 10.222.0.27 () {56 vars in 1374 bytes} [Fri Aug 5 19:49:51 2022] GET /identity/v3/role_assignments?user.id=11001a2ad96f4ec8a35aab2c64376875&scope.project.id=a70a82dcfc574fad90b3f5f00abbba58&role.id=6e2bc1ba438d4eb69923ef4659c181f3 => generated 608 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:49:51.860448 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d0433f-9e0c-468c-b11f-8e4649acd1ae None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:51.868878 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 322/650] 10.222.0.27 () {56 vars in 1109 bytes} [Fri Aug 5 19:49:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:49:51.869881 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37db5f20-a9e7-46bf-b8b4-7340c85c9755 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:51.898145 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:51.949239 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:51.993583 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d0433f-9e0c-468c-b11f-8e4649acd1ae None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:51.997497 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37db5f20-a9e7-46bf-b8b4-7340c85c9755 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:52.036744 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30009b9b-8b5f-4cfa-ad18-c88432c82738 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:52.037464 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30009b9b-8b5f-4cfa-ad18-c88432c82738 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:52.037819 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30009b9b-8b5f-4cfa-ad18-c88432c82738 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:52.038040 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f26b19-c93d-49e4-99e4-44f1730937a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:52.038228 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30009b9b-8b5f-4cfa-ad18-c88432c82738 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:52.039780 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f26b19-c93d-49e4-99e4-44f1730937a1 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:52.040226 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f26b19-c93d-49e4-99e4-44f1730937a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:52.040798 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f26b19-c93d-49e4-99e4-44f1730937a1 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:52.056126 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f26b19-c93d-49e4-99e4-44f1730937a1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 19:49:52.061988 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-79f26b19-c93d-49e4-99e4-44f1730937a1 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 05 19:49:52.061988 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:49:52.061988 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:49:52.061988 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:49:52.061988 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:49:52.061988 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:49:52.061988 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:49:52.061988 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:49:52.061988 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:49:52.061988 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:49:52.061988 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:49:52.061988 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:49:52.061988 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 19:49:52.061988 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 19:49:52.061988 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 19:49:52.061988 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 19:49:52.062901 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 19:49:52.062901 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 19:49:52.062901 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 19:49:52.062901 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 19:49:52.062901 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:49:52.062901 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:49:52.062901 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:49:52.062901 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:49:52.062901 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:49:52.062901 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:49:52.062901 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:49:52.062901 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Aug 05 19:49:52.062901 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:49:52.062901 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:49:52.062901 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:49:52.062901 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:49:52.063739 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:49:52.063739 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:49:52.063739 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:49:52.063739 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:49:52.063739 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 19:49:52.063739 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 19:49:52.063739 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 19:49:52.063739 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 19:49:52.063739 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 19:49:52.063739 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 19:49:52.063739 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 05 19:49:52.063739 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Aug 05 19:49:52.063739 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 329/651] 10.222.0.27 () {56 vars in 1102 bytes} [Fri Aug 5 19:49:51 2022] GET /identity/v3/roles/service => generated 85 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:49:52.064419 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f26b19-c93d-49e4-99e4-44f1730937a1 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:52.125243 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30009b9b-8b5f-4cfa-ad18-c88432c82738 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:52.126771 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30009b9b-8b5f-4cfa-ad18-c88432c82738 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:52.156089 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:52.194568 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f26b19-c93d-49e4-99e4-44f1730937a1 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:52.224820 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:52.236115 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582a60db-7527-470b-ade3-2c1fffa55db5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:52.237297 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582a60db-7527-470b-ade3-2c1fffa55db5 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:52.237588 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582a60db-7527-470b-ade3-2c1fffa55db5 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:52.237870 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582a60db-7527-470b-ade3-2c1fffa55db5 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:52.247362 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-582a60db-7527-470b-ade3-2c1fffa55db5 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:52.249314 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-582a60db-7527-470b-ade3-2c1fffa55db5 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:52.264352 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-582a60db-7527-470b-ade3-2c1fffa55db5 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 05 19:49:52.264352 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:49:52.264352 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:49:52.264352 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:49:52.264352 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:49:52.264352 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:49:52.264352 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:49:52.264352 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:49:52.264352 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:49:52.264352 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:49:52.264352 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:49:52.264352 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:49:52.264352 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 05 19:49:52.264352 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 05 19:49:52.264352 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 05 19:49:52.264352 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 05 19:49:52.265338 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:49:52.265338 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:49:52.265338 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 05 19:49:52.265338 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 19:49:52.265338 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 05 19:49:52.265338 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 19:49:52.265338 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:49:52.265338 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:49:52.265338 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:49:52.265338 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:49:52.265338 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:49:52.265338 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Aug 05 19:49:52.265338 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:49:52.265338 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:49:52.265338 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:49:52.265338 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:49:52.265338 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:49:52.265338 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:49:52.266340 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:49:52.266340 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:49:52.266340 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 05 19:49:52.266340 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 05 19:49:52.266340 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 05 19:49:52.266340 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 05 19:49:52.266340 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 05 19:49:52.266340 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 05 19:49:52.266340 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 05 19:49:52.266340 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Aug 05 19:49:52.266340 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 330/652] 10.222.0.27 () {56 vars in 1099 bytes} [Fri Aug 5 19:49:52 2022] GET /identity/v3/users/glance => generated 84 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:49:52.268048 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582a60db-7527-470b-ade3-2c1fffa55db5 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:52.280599 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 323/653] 10.222.0.27 () {58 vars in 1282 bytes} [Fri Aug 5 19:49:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 408 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 05 19:49:52.281482 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30009b9b-8b5f-4cfa-ad18-c88432c82738 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:52.408871 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582a60db-7527-470b-ade3-2c1fffa55db5 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:52.413145 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30009b9b-8b5f-4cfa-ad18-c88432c82738 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:52.431037 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:52.454733 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a8f239-ab66-4cfc-9967-a22afecbebbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:52.455499 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a8f239-ab66-4cfc-9967-a22afecbebbb None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:52.455781 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a8f239-ab66-4cfc-9967-a22afecbebbb None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:52.456057 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a8f239-ab66-4cfc-9967-a22afecbebbb None admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:52.456334 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af8e1e5-1b9f-448a-901d-6eb77c50e536 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:52.456802 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a8f239-ab66-4cfc-9967-a22afecbebbb None admin] RBAC: Adding query filter params (name=service) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:49:52.457103 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a8f239-ab66-4cfc-9967-a22afecbebbb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:52.457215 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af8e1e5-1b9f-448a-901d-6eb77c50e536 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:52.457456 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af8e1e5-1b9f-448a-901d-6eb77c50e536 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:52.457812 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af8e1e5-1b9f-448a-901d-6eb77c50e536 None admin] PATH_INFO: `/v3/users` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:52.458464 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a8f239-ab66-4cfc-9967-a22afecbebbb None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:52.458733 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af8e1e5-1b9f-448a-901d-6eb77c50e536 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:49:52.459114 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af8e1e5-1b9f-448a-901d-6eb77c50e536 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:52.461207 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af8e1e5-1b9f-448a-901d-6eb77c50e536 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:52.472582 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 331/654] 10.222.0.27 () {56 vars in 1103 bytes} [Fri Aug 5 19:49:52 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:49:52.473482 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a8f239-ab66-4cfc-9967-a22afecbebbb None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:52.487986 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 324/655] 10.222.0.27 () {56 vars in 1137 bytes} [Fri Aug 5 19:49:52 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:49:52.488719 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af8e1e5-1b9f-448a-901d-6eb77c50e536 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:52.539005 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:52.596464 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af8e1e5-1b9f-448a-901d-6eb77c50e536 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:52.606505 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a8f239-ab66-4cfc-9967-a22afecbebbb None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:52.631551 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0008e534-a03e-4cfc-9c1f-2ddd1aa93604 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:52.632188 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0008e534-a03e-4cfc-9c1f-2ddd1aa93604 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:52.632466 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0008e534-a03e-4cfc-9c1f-2ddd1aa93604 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:52.632777 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0008e534-a03e-4cfc-9c1f-2ddd1aa93604 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:52.643245 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0008e534-a03e-4cfc-9c1f-2ddd1aa93604 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:52.645018 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0008e534-a03e-4cfc-9c1f-2ddd1aa93604 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:52.650529 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd666ef-2ed7-4e1f-8f0f-9447de9b3f2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:52.651682 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd666ef-2ed7-4e1f-8f0f-9447de9b3f2d None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:52.652119 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd666ef-2ed7-4e1f-8f0f-9447de9b3f2d None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:52.652893 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd666ef-2ed7-4e1f-8f0f-9447de9b3f2d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:52.664161 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application [None req-0008e534-a03e-4cfc-9c1f-2ddd1aa93604 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:49:52.664161 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:49:52.664161 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:49:52.664161 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:49:52.664161 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:49:52.664161 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:49:52.664161 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:49:52.664161 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:49:52.664161 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:49:52.664161 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:49:52.664161 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:49:52.664161 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:49:52.664161 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 19:49:52.664161 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 19:49:52.664161 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 19:49:52.665092 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 19:49:52.665092 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:49:52.665092 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:49:52.665092 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:49:52.665092 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:49:52.665092 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:49:52.665092 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:49:52.665092 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:49:52.665092 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application with Lock( Aug 05 19:49:52.665092 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:49:52.665092 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:49:52.665092 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:49:52.665092 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:49:52.665092 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:49:52.665092 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:49:52.665092 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:49:52.665092 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:49:52.665985 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 19:49:52.665985 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 19:49:52.665985 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:49:52.665985 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Aug 05 19:49:52.668777 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 325/656] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Aug 5 19:49:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:49:52.669738 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0008e534-a03e-4cfc-9c1f-2ddd1aa93604 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:52.673135 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd666ef-2ed7-4e1f-8f0f-9447de9b3f2d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:52.674933 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd666ef-2ed7-4e1f-8f0f-9447de9b3f2d None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:52.692317 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-5cd666ef-2ed7-4e1f-8f0f-9447de9b3f2d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:49:52.692317 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:49:52.692317 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:49:52.692317 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:49:52.692317 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:49:52.692317 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:49:52.692317 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:49:52.692317 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:49:52.692317 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:49:52.692317 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:49:52.692317 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:49:52.692317 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:49:52.692317 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 19:49:52.692317 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 19:49:52.692317 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 19:49:52.693465 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 19:49:52.693465 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:49:52.693465 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:49:52.693465 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:49:52.693465 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:49:52.693465 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:49:52.693465 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:49:52.693465 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:49:52.693465 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Aug 05 19:49:52.693465 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:49:52.693465 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:49:52.693465 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:49:52.693465 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:49:52.693465 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:49:52.693465 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:49:52.693465 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:49:52.693465 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:49:52.694563 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 19:49:52.694563 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 19:49:52.694563 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:49:52.694563 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Aug 05 19:49:52.695795 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 332/657] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Aug 5 19:49:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:49:52.696760 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd666ef-2ed7-4e1f-8f0f-9447de9b3f2d None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:52.703687 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:52.795796 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0008e534-a03e-4cfc-9c1f-2ddd1aa93604 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:52.815120 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:52.819445 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd666ef-2ed7-4e1f-8f0f-9447de9b3f2d None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:52.833445 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794b1f87-43fe-4a16-acff-3602a6108d9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:52.834399 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794b1f87-43fe-4a16-acff-3602a6108d9c None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:52.834861 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794b1f87-43fe-4a16-acff-3602a6108d9c None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:52.835275 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794b1f87-43fe-4a16-acff-3602a6108d9c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:52.866529 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e56f2f-f797-4d3a-8f5e-de0779b1142f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:52.867291 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e56f2f-f797-4d3a-8f5e-de0779b1142f None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:52.867831 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e56f2f-f797-4d3a-8f5e-de0779b1142f None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:52.868212 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e56f2f-f797-4d3a-8f5e-de0779b1142f None admin] PATH_INFO: `/v3/domains` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:52.869382 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e56f2f-f797-4d3a-8f5e-de0779b1142f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:49:52.869814 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e56f2f-f797-4d3a-8f5e-de0779b1142f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:52.871740 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e56f2f-f797-4d3a-8f5e-de0779b1142f None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:52.893088 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 333/658] 10.222.0.27 () {56 vars in 1109 bytes} [Fri Aug 5 19:49:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:49:52.893088 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e56f2f-f797-4d3a-8f5e-de0779b1142f None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:52.920933 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794b1f87-43fe-4a16-acff-3602a6108d9c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:52.923021 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794b1f87-43fe-4a16-acff-3602a6108d9c None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:52.955529 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:53.006207 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e56f2f-f797-4d3a-8f5e-de0779b1142f None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:53.044758 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d8151e-b664-42dd-bb3f-da189e58646d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:53.046407 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d8151e-b664-42dd-bb3f-da189e58646d None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:53.046904 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d8151e-b664-42dd-bb3f-da189e58646d None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:53.047367 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d8151e-b664-42dd-bb3f-da189e58646d None admin] PATH_INFO: `/v3/domains` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:53.048365 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d8151e-b664-42dd-bb3f-da189e58646d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:49:53.048982 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d8151e-b664-42dd-bb3f-da189e58646d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:53.050528 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d8151e-b664-42dd-bb3f-da189e58646d None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:53.060375 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 326/659] 10.222.0.27 () {58 vars in 1282 bytes} [Fri Aug 5 19:49:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 387 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 05 19:49:53.060375 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794b1f87-43fe-4a16-acff-3602a6108d9c None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:53.069670 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 334/660] 10.222.0.27 () {56 vars in 1109 bytes} [Fri Aug 5 19:49:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:49:53.070917 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d8151e-b664-42dd-bb3f-da189e58646d None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:53.094949 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:53.183633 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794b1f87-43fe-4a16-acff-3602a6108d9c None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:53.193789 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d8151e-b664-42dd-bb3f-da189e58646d None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:53.209762 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:53.222203 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7434462-f950-4e18-b69b-da6d4804cf0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:53.222854 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7434462-f950-4e18-b69b-da6d4804cf0a None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:53.223138 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7434462-f950-4e18-b69b-da6d4804cf0a None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:53.223414 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7434462-f950-4e18-b69b-da6d4804cf0a None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:53.231935 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7434462-f950-4e18-b69b-da6d4804cf0a None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:53.233571 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7434462-f950-4e18-b69b-da6d4804cf0a None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:53.242643 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5035b210-94a6-4013-88b5-4de7aa11ea6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:53.243476 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5035b210-94a6-4013-88b5-4de7aa11ea6a None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:53.243927 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5035b210-94a6-4013-88b5-4de7aa11ea6a None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:53.244349 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5035b210-94a6-4013-88b5-4de7aa11ea6a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:53.246072 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application [None req-f7434462-f950-4e18-b69b-da6d4804cf0a None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 05 19:49:53.246072 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:49:53.246072 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:49:53.246072 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:49:53.246072 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:49:53.246072 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:49:53.246072 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:49:53.246072 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:49:53.246072 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:49:53.246072 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:49:53.246072 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:49:53.246072 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:49:53.246072 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 05 19:49:53.246072 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 05 19:49:53.246072 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 05 19:49:53.246072 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 05 19:49:53.247149 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:49:53.247149 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:49:53.247149 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 05 19:49:53.247149 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 19:49:53.247149 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 05 19:49:53.247149 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 19:49:53.247149 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:49:53.247149 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:49:53.247149 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:49:53.247149 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:49:53.247149 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:49:53.247149 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application with Lock( Aug 05 19:49:53.247149 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:49:53.247149 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:49:53.247149 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:49:53.247149 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:49:53.247149 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:49:53.247149 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:49:53.248834 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:49:53.248834 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:49:53.248834 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 05 19:49:53.248834 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 05 19:49:53.248834 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 05 19:49:53.248834 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 05 19:49:53.248834 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 05 19:49:53.248834 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 05 19:49:53.248834 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 05 19:49:53.248834 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Aug 05 19:49:53.248834 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 327/661] 10.222.0.27 () {56 vars in 1102 bytes} [Fri Aug 5 19:49:53 2022] GET /identity/v3/users/neutron => generated 85 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:49:53.249530 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7434462-f950-4e18-b69b-da6d4804cf0a None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:53.263121 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5035b210-94a6-4013-88b5-4de7aa11ea6a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:53.265712 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5035b210-94a6-4013-88b5-4de7aa11ea6a None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:53.287516 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-5035b210-94a6-4013-88b5-4de7aa11ea6a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:49:53.287516 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:49:53.287516 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:49:53.287516 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:49:53.287516 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:49:53.287516 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:49:53.287516 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:49:53.287516 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:49:53.287516 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:49:53.287516 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:49:53.287516 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:49:53.287516 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:49:53.287516 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 19:49:53.287516 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 19:49:53.287516 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 19:49:53.288439 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 19:49:53.288439 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:49:53.288439 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:49:53.288439 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:49:53.288439 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:49:53.288439 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:49:53.288439 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:49:53.288439 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:49:53.288439 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Aug 05 19:49:53.288439 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:49:53.288439 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:49:53.288439 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:49:53.288439 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:49:53.288439 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:49:53.288439 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:49:53.288439 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:49:53.288439 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:49:53.289374 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 19:49:53.289374 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 19:49:53.289374 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:49:53.289374 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Aug 05 19:49:53.292226 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 335/662] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Aug 5 19:49:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:49:53.293758 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5035b210-94a6-4013-88b5-4de7aa11ea6a None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:53.359776 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7434462-f950-4e18-b69b-da6d4804cf0a None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:53.398108 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:53.398957 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc12d51f-f81a-4b62-a6d3-bed8ece35213 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:53.400251 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc12d51f-f81a-4b62-a6d3-bed8ece35213 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:53.400251 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc12d51f-f81a-4b62-a6d3-bed8ece35213 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:53.400544 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc12d51f-f81a-4b62-a6d3-bed8ece35213 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:53.407982 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc12d51f-f81a-4b62-a6d3-bed8ece35213 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:53.409583 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc12d51f-f81a-4b62-a6d3-bed8ece35213 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:53.421790 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application [None req-cc12d51f-f81a-4b62-a6d3-bed8ece35213 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 19:49:53.421790 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:49:53.421790 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:49:53.421790 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:49:53.421790 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:49:53.421790 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:49:53.421790 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:49:53.421790 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:49:53.421790 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:49:53.421790 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:49:53.421790 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:49:53.421790 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:49:53.421790 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 05 19:49:53.421790 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 05 19:49:53.421790 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 05 19:49:53.422721 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 05 19:49:53.422721 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:49:53.422721 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:49:53.422721 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:49:53.422721 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:49:53.422721 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:49:53.422721 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:49:53.422721 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:49:53.422721 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application with Lock( Aug 05 19:49:53.422721 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:49:53.422721 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:49:53.422721 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:49:53.422721 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:49:53.422721 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:49:53.422721 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:49:53.422721 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:49:53.422721 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:49:53.423618 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 05 19:49:53.423618 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 05 19:49:53.423618 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 05 19:49:53.423618 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 05 19:49:53.423618 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 05 19:49:53.423618 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 05 19:49:53.423618 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 19:49:53.423618 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Aug 05 19:49:53.423618 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 328/663] 10.222.0.27 () {56 vars in 1111 bytes} [Fri Aug 5 19:49:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:49:53.424579 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc12d51f-f81a-4b62-a6d3-bed8ece35213 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:53.445756 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5035b210-94a6-4013-88b5-4de7aa11ea6a None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:53.477206 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:53.496042 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5219f7-4316-4c43-9f52-b6dcada74d62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:53.497405 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5219f7-4316-4c43-9f52-b6dcada74d62 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:53.497697 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5219f7-4316-4c43-9f52-b6dcada74d62 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:53.497975 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5219f7-4316-4c43-9f52-b6dcada74d62 None admin] PATH_INFO: `/v3/users` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:53.499328 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c5219f7-4316-4c43-9f52-b6dcada74d62 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:49:53.499641 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c5219f7-4316-4c43-9f52-b6dcada74d62 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:53.502023 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c5219f7-4316-4c43-9f52-b6dcada74d62 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:53.526596 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 336/664] 10.222.0.27 () {56 vars in 1139 bytes} [Fri Aug 5 19:49:53 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:49:53.527576 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5219f7-4316-4c43-9f52-b6dcada74d62 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:53.538710 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc12d51f-f81a-4b62-a6d3-bed8ece35213 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:53.583484 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4a1c54-7458-4afc-b5d4-592feb59af17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:53.584112 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4a1c54-7458-4afc-b5d4-592feb59af17 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:53.584396 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4a1c54-7458-4afc-b5d4-592feb59af17 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:53.584702 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4a1c54-7458-4afc-b5d4-592feb59af17 None admin] PATH_INFO: `/v3/domains` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:53.586113 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd4a1c54-7458-4afc-b5d4-592feb59af17 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:49:53.586420 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd4a1c54-7458-4afc-b5d4-592feb59af17 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:53.588359 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd4a1c54-7458-4afc-b5d4-592feb59af17 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:53.607761 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 329/665] 10.222.0.27 () {56 vars in 1109 bytes} [Fri Aug 5 19:49:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:49:53.608932 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4a1c54-7458-4afc-b5d4-592feb59af17 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:53.650960 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5219f7-4316-4c43-9f52-b6dcada74d62 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:53.686165 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:53.694474 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b7fa20-c393-46dc-8c2e-1a7165585954 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:53.695385 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b7fa20-c393-46dc-8c2e-1a7165585954 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:53.695890 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b7fa20-c393-46dc-8c2e-1a7165585954 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:53.696420 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b7fa20-c393-46dc-8c2e-1a7165585954 None admin] PATH_INFO: `/v3/projects` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:53.697507 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b7fa20-c393-46dc-8c2e-1a7165585954 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:49:53.697965 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b7fa20-c393-46dc-8c2e-1a7165585954 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:53.699836 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b7fa20-c393-46dc-8c2e-1a7165585954 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:53.713665 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 337/666] 10.222.0.27 () {56 vars in 1148 bytes} [Fri Aug 5 19:49:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:49:53.714860 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b7fa20-c393-46dc-8c2e-1a7165585954 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:53.730573 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4a1c54-7458-4afc-b5d4-592feb59af17 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:53.733602 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:53.768358 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5117f38-ce5c-45d4-a2a0-94d31991c6c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:53.769382 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5117f38-ce5c-45d4-a2a0-94d31991c6c7 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:53.769880 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5117f38-ce5c-45d4-a2a0-94d31991c6c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:53.770357 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5117f38-ce5c-45d4-a2a0-94d31991c6c7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:53.829701 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b7fa20-c393-46dc-8c2e-1a7165585954 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:53.840235 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5117f38-ce5c-45d4-a2a0-94d31991c6c7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:53.841753 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5117f38-ce5c-45d4-a2a0-94d31991c6c7 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:53.866618 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef4785e-a6b7-4a1a-874b-8724b046d2b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:53.867649 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef4785e-a6b7-4a1a-874b-8724b046d2b1 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:53.868196 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef4785e-a6b7-4a1a-874b-8724b046d2b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:53.868608 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef4785e-a6b7-4a1a-874b-8724b046d2b1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:53.876962 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef4785e-a6b7-4a1a-874b-8724b046d2b1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:53.879937 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef4785e-a6b7-4a1a-874b-8724b046d2b1 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:53.898016 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-aef4785e-a6b7-4a1a-874b-8724b046d2b1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 19:49:53.898016 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:49:53.898016 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:49:53.898016 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:49:53.898016 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:49:53.898016 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:49:53.898016 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:49:53.898016 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:49:53.898016 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:49:53.898016 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:49:53.898016 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:49:53.898016 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:49:53.898016 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 05 19:49:53.898016 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 05 19:49:53.898016 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 05 19:49:53.899093 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 05 19:49:53.899093 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:49:53.899093 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:49:53.899093 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:49:53.899093 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:49:53.899093 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:49:53.899093 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:49:53.899093 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:49:53.899093 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Aug 05 19:49:53.899093 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:49:53.899093 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:49:53.899093 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:49:53.899093 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:49:53.899093 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:49:53.899093 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:49:53.899093 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:49:53.899093 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:49:53.900255 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 05 19:49:53.900255 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 05 19:49:53.900255 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 05 19:49:53.900255 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 05 19:49:53.900255 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 05 19:49:53.900255 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 05 19:49:53.900255 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 19:49:53.900255 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Aug 05 19:49:53.900255 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 338/667] 10.222.0.27 () {56 vars in 1111 bytes} [Fri Aug 5 19:49:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:49:53.900255 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef4785e-a6b7-4a1a-874b-8724b046d2b1 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:53.960185 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:53.971542 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 330/668] 10.222.0.27 () {58 vars in 1282 bytes} [Fri Aug 5 19:49:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 05 19:49:53.972913 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5212f4-5ce5-4055-a8f4-c2aa5798f1c3 None None] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:53.973192 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5212f4-5ce5-4055-a8f4-c2aa5798f1c3 None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:53.973471 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5212f4-5ce5-4055-a8f4-c2aa5798f1c3 None None] PATH_INFO: `/` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:53.974749 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 331/669] 10.222.0.27 () {54 vars in 927 bytes} [Fri Aug 5 19:49:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:49:53.975433 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5212f4-5ce5-4055-a8f4-c2aa5798f1c3 None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:53.980026 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:54.016409 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef4785e-a6b7-4a1a-874b-8724b046d2b1 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:54.051966 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f983af-ab2b-43bc-a51f-82e462615498 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:54.052847 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f983af-ab2b-43bc-a51f-82e462615498 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:54.053272 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f983af-ab2b-43bc-a51f-82e462615498 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:54.053704 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f983af-ab2b-43bc-a51f-82e462615498 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:54.054496 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f983af-ab2b-43bc-a51f-82e462615498 None admin] RBAC: Adding query filter params (role.id=6e2bc1ba438d4eb69923ef4659c181f3, scope.project.id=a70a82dcfc574fad90b3f5f00abbba58, user.id=c092a23e14e24265973ff5e899670fc8) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:49:54.054804 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f983af-ab2b-43bc-a51f-82e462615498 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:54.056233 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f983af-ab2b-43bc-a51f-82e462615498 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:54.062712 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 339/670] 10.222.0.27 () {56 vars in 1374 bytes} [Fri Aug 5 19:49:53 2022] GET /identity/v3/role_assignments?user.id=c092a23e14e24265973ff5e899670fc8&scope.project.id=a70a82dcfc574fad90b3f5f00abbba58&role.id=6e2bc1ba438d4eb69923ef4659c181f3 => generated 608 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:49:54.063498 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f983af-ab2b-43bc-a51f-82e462615498 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:54.105144 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5212f4-5ce5-4055-a8f4-c2aa5798f1c3 None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:54.145538 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d339716e-238c-42f7-82b0-97cded502c8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:54.146168 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d339716e-238c-42f7-82b0-97cded502c8b None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:54.146447 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d339716e-238c-42f7-82b0-97cded502c8b None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:54.146720 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d339716e-238c-42f7-82b0-97cded502c8b None admin] PATH_INFO: `/v3/projects` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:54.147486 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d339716e-238c-42f7-82b0-97cded502c8b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:49:54.147794 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d339716e-238c-42f7-82b0-97cded502c8b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:54.149307 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d339716e-238c-42f7-82b0-97cded502c8b None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:54.167394 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 332/671] 10.222.0.27 () {56 vars in 1148 bytes} [Fri Aug 5 19:49:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:49:54.167394 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912b9f8a-de02-4ad6-9b2e-adaa4abbba14 None None] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:54.167394 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912b9f8a-de02-4ad6-9b2e-adaa4abbba14 None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:54.167394 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912b9f8a-de02-4ad6-9b2e-adaa4abbba14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:54.184623 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.auth.core [None req-912b9f8a-de02-4ad6-9b2e-adaa4abbba14 None None] MFA Rules not processed for user `03d1834a64cd4a43829827cb99a72747`. Rule list: `[]` (Enabled: `True`). {{(pid=80364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:49:54.186392 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-912b9f8a-de02-4ad6-9b2e-adaa4abbba14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:49:54.193784 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f983af-ab2b-43bc-a51f-82e462615498 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:54.211363 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:54.238457 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88aed161-a3f6-4058-bbe4-31ee3fc41145 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:54.239109 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88aed161-a3f6-4058-bbe4-31ee3fc41145 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:54.239416 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88aed161-a3f6-4058-bbe4-31ee3fc41145 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:54.239667 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88aed161-a3f6-4058-bbe4-31ee3fc41145 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:54.255065 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88aed161-a3f6-4058-bbe4-31ee3fc41145 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:54.255582 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88aed161-a3f6-4058-bbe4-31ee3fc41145 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:54.263694 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:54.273902 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 333/672] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Aug 5 19:49:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 19:49:54.274055 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-88aed161-a3f6-4058-bbe4-31ee3fc41145 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:49:54.274055 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:49:54.274055 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:49:54.274055 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:49:54.274055 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:49:54.274055 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:49:54.274055 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:49:54.274055 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:49:54.274055 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:49:54.274055 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:49:54.274055 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:49:54.274055 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:49:54.274055 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 19:49:54.274055 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 19:49:54.274055 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 19:49:54.275152 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 19:49:54.275152 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:49:54.275152 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:49:54.275152 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:49:54.275152 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:49:54.275152 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:49:54.275152 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:49:54.275152 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:49:54.275152 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Aug 05 19:49:54.275152 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:49:54.275152 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:49:54.275152 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:49:54.275152 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:49:54.275152 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:49:54.275152 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:49:54.275152 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:49:54.275152 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:49:54.276547 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 19:49:54.276547 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 19:49:54.276547 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:49:54.276547 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Aug 05 19:49:54.276547 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 340/673] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Aug 5 19:49:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:49:54.276848 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c202cd29-61fe-49fb-9ae5-18010a8afa8c None None] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:54.276848 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c202cd29-61fe-49fb-9ae5-18010a8afa8c None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:54.276848 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c202cd29-61fe-49fb-9ae5-18010a8afa8c None None] PATH_INFO: `/` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:54.277025 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88aed161-a3f6-4058-bbe4-31ee3fc41145 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:54.277618 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 334/674] 10.222.0.27 () {54 vars in 927 bytes} [Fri Aug 5 19:49:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:49:54.282412 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c202cd29-61fe-49fb-9ae5-18010a8afa8c None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:54.389338 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88aed161-a3f6-4058-bbe4-31ee3fc41145 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:54.395975 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c202cd29-61fe-49fb-9ae5-18010a8afa8c None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:54.429368 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41614eb-394c-46ba-ba4a-cc045dc0fbbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:54.429987 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41614eb-394c-46ba-ba4a-cc045dc0fbbb None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:54.430282 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41614eb-394c-46ba-ba4a-cc045dc0fbbb None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:54.430546 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41614eb-394c-46ba-ba4a-cc045dc0fbbb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:54.431352 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b41614eb-394c-46ba-ba4a-cc045dc0fbbb None admin] RBAC: Adding query filter params (role.id=6e2bc1ba438d4eb69923ef4659c181f3, scope.project.id=a70a82dcfc574fad90b3f5f00abbba58, user.id=ebf601f787694f33ab81d8021be1a2d9) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:49:54.431645 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b41614eb-394c-46ba-ba4a-cc045dc0fbbb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:54.433130 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b41614eb-394c-46ba-ba4a-cc045dc0fbbb None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:54.434207 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077cf756-ec74-49f0-bae2-a8145628fc7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:54.434828 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077cf756-ec74-49f0-bae2-a8145628fc7d None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:54.435104 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077cf756-ec74-49f0-bae2-a8145628fc7d None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:54.435376 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077cf756-ec74-49f0-bae2-a8145628fc7d None admin] PATH_INFO: `/v3/domains` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:54.436163 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-077cf756-ec74-49f0-bae2-a8145628fc7d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:49:54.436462 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-077cf756-ec74-49f0-bae2-a8145628fc7d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:54.437855 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-077cf756-ec74-49f0-bae2-a8145628fc7d None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:54.440659 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 341/675] 10.222.0.27 () {56 vars in 1374 bytes} [Fri Aug 5 19:49:54 2022] GET /identity/v3/role_assignments?user.id=ebf601f787694f33ab81d8021be1a2d9&scope.project.id=a70a82dcfc574fad90b3f5f00abbba58&role.id=6e2bc1ba438d4eb69923ef4659c181f3 => generated 608 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:49:54.442048 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3db82c-bedb-44f0-861c-625504bd14bd None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:54.442503 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3db82c-bedb-44f0-861c-625504bd14bd None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:54.442941 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3db82c-bedb-44f0-861c-625504bd14bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:54.456037 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 335/676] 10.222.0.27 () {56 vars in 1109 bytes} [Fri Aug 5 19:49:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:49:54.457021 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077cf756-ec74-49f0-bae2-a8145628fc7d None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:54.468060 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-0d3db82c-bedb-44f0-861c-625504bd14bd None None] MFA Rules not processed for user `03d1834a64cd4a43829827cb99a72747`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:49:54.469228 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-0d3db82c-bedb-44f0-861c-625504bd14bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:49:54.497134 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:54.528563 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:54.589141 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077cf756-ec74-49f0-bae2-a8145628fc7d None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:54.632780 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a057953-2f53-42a9-ac21-b92ae1938503 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:54.633453 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a057953-2f53-42a9-ac21-b92ae1938503 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:54.633738 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a057953-2f53-42a9-ac21-b92ae1938503 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:54.634018 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a057953-2f53-42a9-ac21-b92ae1938503 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:54.646687 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a057953-2f53-42a9-ac21-b92ae1938503 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 19:49:54.652585 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application [None req-9a057953-2f53-42a9-ac21-b92ae1938503 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 05 19:49:54.652585 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:49:54.652585 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:49:54.652585 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:49:54.652585 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:49:54.652585 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:49:54.652585 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:49:54.652585 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:49:54.652585 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:49:54.652585 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:49:54.652585 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:49:54.652585 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:49:54.652585 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 19:49:54.652585 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 19:49:54.652585 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 19:49:54.652585 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 19:49:54.653515 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 19:49:54.653515 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 19:49:54.653515 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 19:49:54.653515 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 19:49:54.653515 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:49:54.653515 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:49:54.653515 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:49:54.653515 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:49:54.653515 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:49:54.653515 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:49:54.653515 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:49:54.653515 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application with Lock( Aug 05 19:49:54.653515 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:49:54.653515 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:49:54.653515 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:49:54.653515 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:49:54.654479 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:49:54.654479 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:49:54.654479 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:49:54.654479 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:49:54.654479 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 19:49:54.654479 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 19:49:54.654479 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 19:49:54.654479 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 19:49:54.654479 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 19:49:54.654479 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 19:49:54.654479 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 05 19:49:54.654479 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Aug 05 19:49:54.657388 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 336/677] 10.222.0.27 () {56 vars in 1102 bytes} [Fri Aug 5 19:49:54 2022] GET /identity/v3/roles/service => generated 85 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:49:54.657501 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a057953-2f53-42a9-ac21-b92ae1938503 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:54.696551 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 342/678] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Aug 5 19:49:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 248 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 19:49:54.696551 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3db82c-bedb-44f0-861c-625504bd14bd None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:54.785717 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a057953-2f53-42a9-ac21-b92ae1938503 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:54.800312 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:54.823601 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3db82c-bedb-44f0-861c-625504bd14bd None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:54.824624 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71c1fe4-da89-42ac-a0ea-7375eae35825 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:54.825270 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71c1fe4-da89-42ac-a0ea-7375eae35825 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:54.825545 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71c1fe4-da89-42ac-a0ea-7375eae35825 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:54.825860 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71c1fe4-da89-42ac-a0ea-7375eae35825 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:54.834777 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71c1fe4-da89-42ac-a0ea-7375eae35825 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:54.836231 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71c1fe4-da89-42ac-a0ea-7375eae35825 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:54.848991 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application [None req-d71c1fe4-da89-42ac-a0ea-7375eae35825 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 19:49:54.848991 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:49:54.848991 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:49:54.848991 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:49:54.848991 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:49:54.848991 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:49:54.848991 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:49:54.848991 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:49:54.848991 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:49:54.848991 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:49:54.848991 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:49:54.848991 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:49:54.848991 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 05 19:49:54.848991 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 05 19:49:54.848991 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 05 19:49:54.849943 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 05 19:49:54.849943 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:49:54.849943 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:49:54.849943 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:49:54.849943 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:49:54.849943 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:49:54.849943 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:49:54.849943 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:49:54.849943 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application with Lock( Aug 05 19:49:54.849943 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:49:54.849943 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:49:54.849943 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:49:54.849943 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:49:54.849943 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:49:54.849943 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:49:54.849943 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:49:54.849943 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:49:54.850832 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 05 19:49:54.850832 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 05 19:49:54.850832 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 05 19:49:54.850832 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 05 19:49:54.850832 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 05 19:49:54.850832 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 05 19:49:54.850832 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 19:49:54.850832 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Aug 05 19:49:54.850832 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 337/679] 10.222.0.27 () {56 vars in 1111 bytes} [Fri Aug 5 19:49:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:49:54.851315 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71c1fe4-da89-42ac-a0ea-7375eae35825 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:54.868025 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d2bd77-d1fb-452c-8f7e-a48025b3b7ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:54.869010 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d2bd77-d1fb-452c-8f7e-a48025b3b7ca None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:54.869510 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d2bd77-d1fb-452c-8f7e-a48025b3b7ca None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:54.869997 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d2bd77-d1fb-452c-8f7e-a48025b3b7ca None admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:54.870920 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d2bd77-d1fb-452c-8f7e-a48025b3b7ca None admin] RBAC: Adding query filter params (name=service) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:49:54.871424 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d2bd77-d1fb-452c-8f7e-a48025b3b7ca None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:54.873601 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d2bd77-d1fb-452c-8f7e-a48025b3b7ca None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:54.882668 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 343/680] 10.222.0.27 () {56 vars in 1103 bytes} [Fri Aug 5 19:49:54 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:49:54.883963 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d2bd77-d1fb-452c-8f7e-a48025b3b7ca None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:54.936879 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:54.964849 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71c1fe4-da89-42ac-a0ea-7375eae35825 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:54.999779 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d2bd77-d1fb-452c-8f7e-a48025b3b7ca None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:55.000833 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc6539f-e314-48f5-bc4b-78e4911818fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:55.001537 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc6539f-e314-48f5-bc4b-78e4911818fa None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:55.001817 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc6539f-e314-48f5-bc4b-78e4911818fa None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:55.002090 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc6539f-e314-48f5-bc4b-78e4911818fa None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:55.013238 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc6539f-e314-48f5-bc4b-78e4911818fa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 19:49:55.018736 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application [None req-ebc6539f-e314-48f5-bc4b-78e4911818fa None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 19:49:55.018736 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:49:55.018736 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:49:55.018736 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:49:55.018736 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:49:55.018736 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:49:55.018736 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:49:55.018736 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:49:55.018736 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:49:55.018736 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:49:55.018736 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:49:55.018736 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:49:55.018736 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 19:49:55.018736 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 19:49:55.018736 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 19:49:55.018736 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 19:49:55.019687 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 19:49:55.019687 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 19:49:55.019687 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 19:49:55.019687 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 19:49:55.019687 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:49:55.019687 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:49:55.019687 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:49:55.019687 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:49:55.019687 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:49:55.019687 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:49:55.019687 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:49:55.019687 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application with Lock( Aug 05 19:49:55.019687 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:49:55.019687 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:49:55.019687 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:49:55.019687 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:49:55.020551 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:49:55.020551 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:49:55.020551 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:49:55.020551 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:49:55.020551 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 19:49:55.020551 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 19:49:55.020551 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 19:49:55.020551 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 19:49:55.020551 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 19:49:55.020551 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 19:49:55.020551 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 19:49:55.020551 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Aug 05 19:49:55.020551 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 338/681] 10.222.0.27 () {56 vars in 1096 bytes} [Fri Aug 5 19:49:54 2022] GET /identity/v3/roles/admin => generated 83 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:49:55.021254 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc6539f-e314-48f5-bc4b-78e4911818fa None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:55.044102 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5a130b-8a13-4679-9cd1-ac89977945ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:55.045674 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5a130b-8a13-4679-9cd1-ac89977945ef None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:55.046159 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5a130b-8a13-4679-9cd1-ac89977945ef None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:55.046632 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5a130b-8a13-4679-9cd1-ac89977945ef None admin] PATH_INFO: `/v3/projects` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:55.047726 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa5a130b-8a13-4679-9cd1-ac89977945ef None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:49:55.048352 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa5a130b-8a13-4679-9cd1-ac89977945ef None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:55.050127 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa5a130b-8a13-4679-9cd1-ac89977945ef None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:55.057296 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:55.069922 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 344/682] 10.222.0.27 () {56 vars in 1148 bytes} [Fri Aug 5 19:49:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:49:55.071976 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5a130b-8a13-4679-9cd1-ac89977945ef None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:55.128233 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc6539f-e314-48f5-bc4b-78e4911818fa None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:55.161083 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:55.164793 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c998849-4d69-490e-8ff8-f01a70f94647 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:55.165957 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c998849-4d69-490e-8ff8-f01a70f94647 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:55.166242 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c998849-4d69-490e-8ff8-f01a70f94647 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:55.166517 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c998849-4d69-490e-8ff8-f01a70f94647 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:55.177972 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c998849-4d69-490e-8ff8-f01a70f94647 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:55.179425 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c998849-4d69-490e-8ff8-f01a70f94647 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:55.184084 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5a130b-8a13-4679-9cd1-ac89977945ef None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:55.199340 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application [None req-2c998849-4d69-490e-8ff8-f01a70f94647 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:49:55.199340 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:49:55.199340 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:49:55.199340 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:49:55.199340 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:49:55.199340 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:49:55.199340 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:49:55.199340 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:49:55.199340 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:49:55.199340 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:49:55.199340 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:49:55.199340 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:49:55.199340 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 19:49:55.199340 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 19:49:55.199340 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 19:49:55.200284 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 19:49:55.200284 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:49:55.200284 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:49:55.200284 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:49:55.200284 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:49:55.200284 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:49:55.200284 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:49:55.200284 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:49:55.200284 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application with Lock( Aug 05 19:49:55.200284 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:49:55.200284 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:49:55.200284 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:49:55.200284 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:49:55.200284 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:49:55.200284 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:49:55.200284 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:49:55.200284 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:49:55.201753 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 19:49:55.201753 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 19:49:55.201753 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:49:55.201753 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Aug 05 19:49:55.201753 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 339/683] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Aug 5 19:49:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:49:55.202043 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c998849-4d69-490e-8ff8-f01a70f94647 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:55.223152 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea23c1c0-82d7-4456-8ffb-aadd4c50cedb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:55.224127 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea23c1c0-82d7-4456-8ffb-aadd4c50cedb None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:55.224684 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea23c1c0-82d7-4456-8ffb-aadd4c50cedb None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:55.225061 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea23c1c0-82d7-4456-8ffb-aadd4c50cedb None admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:55.225885 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea23c1c0-82d7-4456-8ffb-aadd4c50cedb None admin] RBAC: Adding query filter params (name=admin) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:49:55.226290 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea23c1c0-82d7-4456-8ffb-aadd4c50cedb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:55.227767 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea23c1c0-82d7-4456-8ffb-aadd4c50cedb None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:55.237544 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 345/684] 10.222.0.27 () {56 vars in 1099 bytes} [Fri Aug 5 19:49:55 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:49:55.238455 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea23c1c0-82d7-4456-8ffb-aadd4c50cedb None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:55.314839 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:55.331468 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c998849-4d69-490e-8ff8-f01a70f94647 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:55.352132 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea23c1c0-82d7-4456-8ffb-aadd4c50cedb None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:55.377081 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a5a592-db9c-4cab-85a1-53c25bbbfc3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:55.377734 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a5a592-db9c-4cab-85a1-53c25bbbfc3b None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:55.378112 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a5a592-db9c-4cab-85a1-53c25bbbfc3b None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:55.378736 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a5a592-db9c-4cab-85a1-53c25bbbfc3b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:55.379154 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a5a592-db9c-4cab-85a1-53c25bbbfc3b None admin] RBAC: Adding query filter params (role.id=6e2bc1ba438d4eb69923ef4659c181f3, scope.project.id=a70a82dcfc574fad90b3f5f00abbba58, user.id=6e632e7fe71448c29273ac65b5899aca) {{(pid=80364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:49:55.379462 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a5a592-db9c-4cab-85a1-53c25bbbfc3b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:55.381050 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a5a592-db9c-4cab-85a1-53c25bbbfc3b None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:55.391921 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9408b6-2bca-4a52-8e34-171598563171 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:55.392951 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9408b6-2bca-4a52-8e34-171598563171 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:55.393461 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9408b6-2bca-4a52-8e34-171598563171 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:55.393931 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9408b6-2bca-4a52-8e34-171598563171 None admin] PATH_INFO: `/v3/domains` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:55.394878 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a9408b6-2bca-4a52-8e34-171598563171 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:49:55.395391 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a9408b6-2bca-4a52-8e34-171598563171 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:55.396945 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 340/685] 10.222.0.27 () {56 vars in 1374 bytes} [Fri Aug 5 19:49:55 2022] GET /identity/v3/role_assignments?user.id=6e632e7fe71448c29273ac65b5899aca&scope.project.id=a70a82dcfc574fad90b3f5f00abbba58&role.id=6e2bc1ba438d4eb69923ef4659c181f3 => generated 608 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:49:55.397341 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a9408b6-2bca-4a52-8e34-171598563171 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:55.397911 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a5a592-db9c-4cab-85a1-53c25bbbfc3b None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:55.411920 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 346/686] 10.222.0.27 () {56 vars in 1109 bytes} [Fri Aug 5 19:49:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:49:55.421799 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9408b6-2bca-4a52-8e34-171598563171 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:55.426145 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:55.514402 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a5a592-db9c-4cab-85a1-53c25bbbfc3b None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:55.561428 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7996cc-9c29-4c9b-8cea-3b5d3737f50f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:55.562524 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7996cc-9c29-4c9b-8cea-3b5d3737f50f None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:55.563000 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7996cc-9c29-4c9b-8cea-3b5d3737f50f None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:55.563468 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7996cc-9c29-4c9b-8cea-3b5d3737f50f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:55.584463 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac7996cc-9c29-4c9b-8cea-3b5d3737f50f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:55.590616 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac7996cc-9c29-4c9b-8cea-3b5d3737f50f None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:55.617487 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application [None req-ac7996cc-9c29-4c9b-8cea-3b5d3737f50f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:49:55.617487 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:49:55.617487 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:49:55.617487 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:49:55.617487 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:49:55.617487 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:49:55.617487 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:49:55.617487 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:49:55.617487 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:49:55.617487 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:49:55.617487 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:49:55.617487 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:49:55.617487 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 19:49:55.617487 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 19:49:55.617487 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 19:49:55.618860 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 19:49:55.618860 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:49:55.618860 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:49:55.618860 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:49:55.618860 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:49:55.618860 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:49:55.618860 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:49:55.618860 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:49:55.618860 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application with Lock( Aug 05 19:49:55.618860 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:49:55.618860 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:49:55.618860 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:49:55.618860 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:49:55.618860 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:49:55.618860 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:49:55.618860 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:49:55.618860 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:49:55.619774 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 19:49:55.619774 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 19:49:55.619774 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:49:55.619774 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Aug 05 19:49:55.626641 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 341/687] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Aug 5 19:49:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:49:55.629851 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9408b6-2bca-4a52-8e34-171598563171 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:55.632595 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7996cc-9c29-4c9b-8cea-3b5d3737f50f None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:55.648486 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:55.684598 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc57a734-eef1-4a16-9e77-bb7975d26c1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:55.685951 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc57a734-eef1-4a16-9e77-bb7975d26c1c None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:55.686242 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc57a734-eef1-4a16-9e77-bb7975d26c1c None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:55.686514 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc57a734-eef1-4a16-9e77-bb7975d26c1c None admin] PATH_INFO: `/v3/users/swift` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:55.696775 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc57a734-eef1-4a16-9e77-bb7975d26c1c None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:55.698326 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc57a734-eef1-4a16-9e77-bb7975d26c1c None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:55.712132 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-fc57a734-eef1-4a16-9e77-bb7975d26c1c None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 05 19:49:55.712132 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:49:55.712132 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:49:55.712132 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:49:55.712132 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:49:55.712132 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:49:55.712132 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:49:55.712132 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:49:55.712132 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:49:55.712132 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:49:55.712132 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:49:55.712132 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:49:55.712132 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 05 19:49:55.712132 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 05 19:49:55.712132 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 05 19:49:55.712132 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 05 19:49:55.713177 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:49:55.713177 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:49:55.713177 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 05 19:49:55.713177 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 19:49:55.713177 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 05 19:49:55.713177 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 19:49:55.713177 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:49:55.713177 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:49:55.713177 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:49:55.713177 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:49:55.713177 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:49:55.713177 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Aug 05 19:49:55.713177 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:49:55.713177 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:49:55.713177 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:49:55.713177 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:49:55.713177 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:49:55.713177 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:49:55.714116 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:49:55.714116 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:49:55.714116 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 05 19:49:55.714116 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 05 19:49:55.714116 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 05 19:49:55.714116 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 05 19:49:55.714116 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 05 19:49:55.714116 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 05 19:49:55.714116 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 05 19:49:55.714116 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Aug 05 19:49:55.714116 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 347/688] 10.222.0.27 () {56 vars in 1096 bytes} [Fri Aug 5 19:49:55 2022] GET /identity/v3/users/swift => generated 83 bytes in 292 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:49:55.722806 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc57a734-eef1-4a16-9e77-bb7975d26c1c None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:55.771201 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7996cc-9c29-4c9b-8cea-3b5d3737f50f None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:55.800005 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:55.816775 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a398e6-10fb-430b-b6d9-6e669456a228 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:55.817559 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a398e6-10fb-430b-b6d9-6e669456a228 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:55.817939 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a398e6-10fb-430b-b6d9-6e669456a228 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:55.818310 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a398e6-10fb-430b-b6d9-6e669456a228 None admin] PATH_INFO: `/v3/domains` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:55.819566 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a398e6-10fb-430b-b6d9-6e669456a228 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:49:55.819976 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a398e6-10fb-430b-b6d9-6e669456a228 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:55.822072 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a398e6-10fb-430b-b6d9-6e669456a228 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:55.839891 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 342/689] 10.222.0.27 () {56 vars in 1109 bytes} [Fri Aug 5 19:49:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:49:55.845549 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a398e6-10fb-430b-b6d9-6e669456a228 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:55.848168 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc57a734-eef1-4a16-9e77-bb7975d26c1c None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:55.884488 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7a298d-41ef-44b5-83a5-7d68cdf74a0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:55.885496 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7a298d-41ef-44b5-83a5-7d68cdf74a0b None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:55.885951 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7a298d-41ef-44b5-83a5-7d68cdf74a0b None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:55.886394 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7a298d-41ef-44b5-83a5-7d68cdf74a0b None admin] PATH_INFO: `/v3/users` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:55.887359 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f7a298d-41ef-44b5-83a5-7d68cdf74a0b None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:49:55.887877 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f7a298d-41ef-44b5-83a5-7d68cdf74a0b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:55.890313 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f7a298d-41ef-44b5-83a5-7d68cdf74a0b None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:55.909084 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 348/690] 10.222.0.27 () {56 vars in 1135 bytes} [Fri Aug 5 19:49:55 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:49:55.914974 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7a298d-41ef-44b5-83a5-7d68cdf74a0b None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:55.916845 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:55.956817 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a398e6-10fb-430b-b6d9-6e669456a228 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:56.005831 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb49813-28ea-44fc-a7d0-4395b018133e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:56.006657 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb49813-28ea-44fc-a7d0-4395b018133e None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:56.007024 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb49813-28ea-44fc-a7d0-4395b018133e None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:56.007400 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb49813-28ea-44fc-a7d0-4395b018133e None admin] PATH_INFO: `/v3/users/nova` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:56.016809 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb49813-28ea-44fc-a7d0-4395b018133e None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:56.018424 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb49813-28ea-44fc-a7d0-4395b018133e None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:56.025041 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7a298d-41ef-44b5-83a5-7d68cdf74a0b None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:56.035274 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application [None req-2bb49813-28ea-44fc-a7d0-4395b018133e None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 05 19:49:56.035274 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:49:56.035274 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:49:56.035274 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:49:56.035274 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:49:56.035274 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:49:56.035274 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:49:56.035274 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:49:56.035274 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:49:56.035274 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:49:56.035274 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:49:56.035274 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:49:56.035274 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 05 19:49:56.035274 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 05 19:49:56.035274 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 05 19:49:56.035274 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 05 19:49:56.036366 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:49:56.036366 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:49:56.036366 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 05 19:49:56.036366 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 19:49:56.036366 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 05 19:49:56.036366 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 19:49:56.036366 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:49:56.036366 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:49:56.036366 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:49:56.036366 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:49:56.036366 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:49:56.036366 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application with Lock( Aug 05 19:49:56.036366 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:49:56.036366 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:49:56.036366 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:49:56.036366 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:49:56.036366 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:49:56.036366 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:49:56.037338 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:49:56.037338 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:49:56.037338 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 05 19:49:56.037338 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 05 19:49:56.037338 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 05 19:49:56.037338 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 05 19:49:56.037338 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 05 19:49:56.037338 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 05 19:49:56.037338 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 05 19:49:56.037338 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Aug 05 19:49:56.045468 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 343/691] 10.222.0.27 () {56 vars in 1093 bytes} [Fri Aug 5 19:49:55 2022] GET /identity/v3/users/nova => generated 82 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 05 19:49:56.050950 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb49813-28ea-44fc-a7d0-4395b018133e None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:56.054604 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:56.081802 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9f0f7b-c51a-4604-b6ea-477604ff96dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:56.082787 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9f0f7b-c51a-4604-b6ea-477604ff96dd None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:56.083262 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9f0f7b-c51a-4604-b6ea-477604ff96dd None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:56.083751 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9f0f7b-c51a-4604-b6ea-477604ff96dd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:56.165634 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc9f0f7b-c51a-4604-b6ea-477604ff96dd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:56.167464 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc9f0f7b-c51a-4604-b6ea-477604ff96dd None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:56.190791 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb49813-28ea-44fc-a7d0-4395b018133e None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:56.225832 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd97f6b-5f20-4117-979d-231a8e2657c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:56.226462 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd97f6b-5f20-4117-979d-231a8e2657c5 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:56.226742 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd97f6b-5f20-4117-979d-231a8e2657c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:56.227142 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd97f6b-5f20-4117-979d-231a8e2657c5 None admin] PATH_INFO: `/v3/users` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:56.227950 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd97f6b-5f20-4117-979d-231a8e2657c5 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:49:56.228260 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd97f6b-5f20-4117-979d-231a8e2657c5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:56.229879 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd97f6b-5f20-4117-979d-231a8e2657c5 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:56.233558 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:56.254091 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 344/692] 10.222.0.27 () {56 vars in 1133 bytes} [Fri Aug 5 19:49:56 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:49:56.255019 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd97f6b-5f20-4117-979d-231a8e2657c5 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:56.339059 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 349/693] 10.222.0.27 () {58 vars in 1282 bytes} [Fri Aug 5 19:49:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 424 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 05 19:49:56.340838 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab51329-26de-483b-81a1-9579a361be95 None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:56.340838 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab51329-26de-483b-81a1-9579a361be95 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:56.341048 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab51329-26de-483b-81a1-9579a361be95 None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:56.343670 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 350/694] 10.222.0.27 () {54 vars in 927 bytes} [Fri Aug 5 19:49:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:49:56.344921 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab51329-26de-483b-81a1-9579a361be95 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:56.383385 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd97f6b-5f20-4117-979d-231a8e2657c5 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:56.388961 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:56.433634 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a09eee-6478-49dd-8b0a-470b815bb186 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:56.434804 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a09eee-6478-49dd-8b0a-470b815bb186 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:56.435185 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a09eee-6478-49dd-8b0a-470b815bb186 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:56.436091 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a09eee-6478-49dd-8b0a-470b815bb186 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:56.486600 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab51329-26de-483b-81a1-9579a361be95 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:56.505069 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:56.525245 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a09eee-6478-49dd-8b0a-470b815bb186 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:56.527080 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a09eee-6478-49dd-8b0a-470b815bb186 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:56.527394 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9cafd6-d10b-44a1-81f6-16c56dbc63bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:56.528084 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9cafd6-d10b-44a1-81f6-16c56dbc63bb None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:56.528368 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9cafd6-d10b-44a1-81f6-16c56dbc63bb None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:56.528705 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9cafd6-d10b-44a1-81f6-16c56dbc63bb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:56.543768 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db9cafd6-d10b-44a1-81f6-16c56dbc63bb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:56.545809 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db9cafd6-d10b-44a1-81f6-16c56dbc63bb None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:56.562647 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-db9cafd6-d10b-44a1-81f6-16c56dbc63bb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:49:56.562647 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:49:56.562647 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:49:56.562647 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:49:56.562647 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:49:56.562647 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:49:56.562647 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:49:56.562647 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:49:56.562647 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:49:56.562647 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:49:56.562647 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:49:56.562647 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:49:56.562647 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 19:49:56.562647 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 19:49:56.562647 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 19:49:56.563555 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 19:49:56.563555 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:49:56.563555 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:49:56.563555 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:49:56.563555 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:49:56.563555 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:49:56.563555 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:49:56.563555 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:49:56.563555 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Aug 05 19:49:56.563555 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:49:56.563555 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:49:56.563555 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:49:56.563555 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:49:56.563555 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:49:56.563555 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:49:56.563555 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:49:56.563555 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:49:56.564440 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 19:49:56.564440 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 19:49:56.564440 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:49:56.564440 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Aug 05 19:49:56.564440 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 351/695] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Aug 5 19:49:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 05 19:49:56.566532 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f083b8ce-1ccf-4d58-a02f-c7df079547b6 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:56.566823 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f083b8ce-1ccf-4d58-a02f-c7df079547b6 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:56.567103 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f083b8ce-1ccf-4d58-a02f-c7df079547b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:56.586555 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-f083b8ce-1ccf-4d58-a02f-c7df079547b6 None None] MFA Rules not processed for user `03d1834a64cd4a43829827cb99a72747`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:49:56.587881 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-f083b8ce-1ccf-4d58-a02f-c7df079547b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:49:56.660693 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 345/696] 10.222.0.27 () {58 vars in 1282 bytes} [Fri Aug 5 19:49:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 401 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 05 19:49:56.660693 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a09eee-6478-49dd-8b0a-470b815bb186 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:56.685536 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 352/697] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Aug 5 19:49:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 19:49:56.687408 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f083b8ce-1ccf-4d58-a02f-c7df079547b6 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:56.692053 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:56.767611 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:56.787697 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a09eee-6478-49dd-8b0a-470b815bb186 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:56.806764 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f083b8ce-1ccf-4d58-a02f-c7df079547b6 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:56.823391 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24a3c0b-f364-450f-a583-97260f07fffb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:56.824009 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24a3c0b-f364-450f-a583-97260f07fffb None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:56.824278 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24a3c0b-f364-450f-a583-97260f07fffb None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:56.824574 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24a3c0b-f364-450f-a583-97260f07fffb None admin] PATH_INFO: `/v3/domains` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:56.825265 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24a3c0b-f364-450f-a583-97260f07fffb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:49:56.825570 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24a3c0b-f364-450f-a583-97260f07fffb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:56.826996 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24a3c0b-f364-450f-a583-97260f07fffb None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:56.838406 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 346/698] 10.222.0.27 () {56 vars in 1109 bytes} [Fri Aug 5 19:49:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:49:56.839227 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24a3c0b-f364-450f-a583-97260f07fffb None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:56.849959 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2160d4b4-0b18-4602-95c3-b2863da4f2d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:56.850604 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2160d4b4-0b18-4602-95c3-b2863da4f2d0 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:56.850883 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2160d4b4-0b18-4602-95c3-b2863da4f2d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:56.851322 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2160d4b4-0b18-4602-95c3-b2863da4f2d0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:56.863029 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2160d4b4-0b18-4602-95c3-b2863da4f2d0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:56.864476 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2160d4b4-0b18-4602-95c3-b2863da4f2d0 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:56.880203 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-2160d4b4-0b18-4602-95c3-b2863da4f2d0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:49:56.880203 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:49:56.880203 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:49:56.880203 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:49:56.880203 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:49:56.880203 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:49:56.880203 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:49:56.880203 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:49:56.880203 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:49:56.880203 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:49:56.880203 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:49:56.880203 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:49:56.880203 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 19:49:56.880203 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 19:49:56.880203 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 19:49:56.881359 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 19:49:56.881359 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:49:56.881359 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:49:56.881359 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:49:56.881359 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:49:56.881359 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:49:56.881359 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:49:56.881359 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:49:56.881359 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Aug 05 19:49:56.881359 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:49:56.881359 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:49:56.881359 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:49:56.881359 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:49:56.881359 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:49:56.881359 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:49:56.881359 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:49:56.881359 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:49:56.882430 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 19:49:56.882430 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 19:49:56.882430 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:49:56.882430 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Aug 05 19:49:56.885914 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 353/699] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Aug 5 19:49:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:49:56.887121 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da8b509-6ed5-47a0-b4c8-0451c90f0d88 None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:56.887410 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da8b509-6ed5-47a0-b4c8-0451c90f0d88 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:56.887721 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da8b509-6ed5-47a0-b4c8-0451c90f0d88 None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:56.889711 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 354/700] 10.222.0.27 () {54 vars in 927 bytes} [Fri Aug 5 19:49:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:49:56.890270 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da8b509-6ed5-47a0-b4c8-0451c90f0d88 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:56.953350 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24a3c0b-f364-450f-a583-97260f07fffb None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:56.954284 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:56.995709 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db57fcf-665b-4d0c-a88b-40bf82443a16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:56.996599 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db57fcf-665b-4d0c-a88b-40bf82443a16 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:56.996969 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db57fcf-665b-4d0c-a88b-40bf82443a16 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:56.997384 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db57fcf-665b-4d0c-a88b-40bf82443a16 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:56.998666 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db57fcf-665b-4d0c-a88b-40bf82443a16 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:57.000180 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db57fcf-665b-4d0c-a88b-40bf82443a16 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:57.006997 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da8b509-6ed5-47a0-b4c8-0451c90f0d88 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:57.014460 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application [None req-4db57fcf-665b-4d0c-a88b-40bf82443a16 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 05 19:49:57.014460 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:49:57.014460 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 05 19:49:57.014460 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 05 19:49:57.014460 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 05 19:49:57.014460 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 05 19:49:57.014460 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 05 19:49:57.014460 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 05 19:49:57.014460 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 05 19:49:57.014460 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Aug 05 19:49:57.014460 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 05 19:49:57.014460 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Aug 05 19:49:57.014460 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:49:57.014460 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:49:57.014460 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:49:57.014460 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:49:57.015598 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:49:57.015598 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:49:57.015598 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:49:57.015598 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:49:57.015598 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:49:57.015598 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:49:57.015598 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:49:57.015598 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 05 19:49:57.015598 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 05 19:49:57.015598 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 05 19:49:57.015598 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 05 19:49:57.015598 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:49:57.015598 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:49:57.015598 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:49:57.015598 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:49:57.015598 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:49:57.016594 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:49:57.016594 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:49:57.016594 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application with Lock( Aug 05 19:49:57.016594 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:49:57.016594 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:49:57.016594 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:49:57.016594 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:49:57.016594 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:49:57.016594 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:49:57.016594 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:49:57.016594 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:49:57.016594 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 05 19:49:57.016594 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 05 19:49:57.016594 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 05 19:49:57.016594 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Aug 05 19:49:57.016594 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 347/701] 10.222.0.27 () {56 vars in 1129 bytes} [Fri Aug 5 19:49:56 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:49:57.017533 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db57fcf-665b-4d0c-a88b-40bf82443a16 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:57.021426 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:57.051924 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41eb7c99-3250-452a-9d53-cc4709bf29a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:57.052945 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41eb7c99-3250-452a-9d53-cc4709bf29a5 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:57.053299 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41eb7c99-3250-452a-9d53-cc4709bf29a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:57.053643 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41eb7c99-3250-452a-9d53-cc4709bf29a5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:57.062364 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41eb7c99-3250-452a-9d53-cc4709bf29a5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:57.063886 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41eb7c99-3250-452a-9d53-cc4709bf29a5 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:57.077063 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-41eb7c99-3250-452a-9d53-cc4709bf29a5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 19:49:57.077063 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:49:57.077063 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:49:57.077063 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:49:57.077063 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:49:57.077063 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:49:57.077063 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:49:57.077063 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:49:57.077063 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:49:57.077063 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:49:57.077063 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:49:57.077063 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:49:57.077063 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 05 19:49:57.077063 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 05 19:49:57.077063 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 05 19:49:57.078188 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 05 19:49:57.078188 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:49:57.078188 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:49:57.078188 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:49:57.078188 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:49:57.078188 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:49:57.078188 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:49:57.078188 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:49:57.078188 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Aug 05 19:49:57.078188 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:49:57.078188 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:49:57.078188 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:49:57.078188 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:49:57.078188 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:49:57.078188 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:49:57.078188 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:49:57.078188 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:49:57.079077 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 05 19:49:57.079077 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 05 19:49:57.079077 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 05 19:49:57.079077 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 05 19:49:57.079077 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 05 19:49:57.079077 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 05 19:49:57.079077 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 19:49:57.079077 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Aug 05 19:49:57.079077 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 355/702] 10.222.0.27 () {56 vars in 1111 bytes} [Fri Aug 5 19:49:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:49:57.079782 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9fd9b1-d963-42f5-9b9a-88d44562d60a None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:57.080069 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9fd9b1-d963-42f5-9b9a-88d44562d60a None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:57.080345 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9fd9b1-d963-42f5-9b9a-88d44562d60a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:57.098364 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-7e9fd9b1-d963-42f5-9b9a-88d44562d60a None None] MFA Rules not processed for user `03d1834a64cd4a43829827cb99a72747`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:49:57.099565 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-7e9fd9b1-d963-42f5-9b9a-88d44562d60a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:49:57.139234 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db57fcf-665b-4d0c-a88b-40bf82443a16 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:57.177280 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 356/703] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Aug 5 19:49:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 19:49:57.178136 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9fd9b1-d963-42f5-9b9a-88d44562d60a None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:57.179815 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d7407b-df9b-4a0f-8aa1-7c44f7d41e4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:57.180766 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d7407b-df9b-4a0f-8aa1-7c44f7d41e4b None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:57.181102 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d7407b-df9b-4a0f-8aa1-7c44f7d41e4b None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:57.181432 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d7407b-df9b-4a0f-8aa1-7c44f7d41e4b None admin] PATH_INFO: `/v3/domains` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:57.182187 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d7407b-df9b-4a0f-8aa1-7c44f7d41e4b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:49:57.182552 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d7407b-df9b-4a0f-8aa1-7c44f7d41e4b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:57.183973 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d7407b-df9b-4a0f-8aa1-7c44f7d41e4b None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:57.197465 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 348/704] 10.222.0.27 () {56 vars in 1109 bytes} [Fri Aug 5 19:49:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:49:57.202916 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d7407b-df9b-4a0f-8aa1-7c44f7d41e4b None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:57.234002 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:57.273051 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:57.297683 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9fd9b1-d963-42f5-9b9a-88d44562d60a None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:57.314865 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d7407b-df9b-4a0f-8aa1-7c44f7d41e4b None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:57.334472 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbca9646-45c3-4015-b2e3-08cdce93d999 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:57.335213 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbca9646-45c3-4015-b2e3-08cdce93d999 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:57.335487 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbca9646-45c3-4015-b2e3-08cdce93d999 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:57.335758 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbca9646-45c3-4015-b2e3-08cdce93d999 None admin] PATH_INFO: `/v3/services` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:57.336444 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbca9646-45c3-4015-b2e3-08cdce93d999 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:49:57.336899 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbca9646-45c3-4015-b2e3-08cdce93d999 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:57.338383 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbca9646-45c3-4015-b2e3-08cdce93d999 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:57.343638 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 357/705] 10.222.0.27 () {56 vars in 1124 bytes} [Fri Aug 5 19:49:57 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:49:57.348817 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbca9646-45c3-4015-b2e3-08cdce93d999 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:57.349645 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257b45c5-7d6c-468b-af86-4b3567ba675b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:57.350380 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257b45c5-7d6c-468b-af86-4b3567ba675b None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:57.350759 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257b45c5-7d6c-468b-af86-4b3567ba675b None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:57.351160 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257b45c5-7d6c-468b-af86-4b3567ba675b None admin] PATH_INFO: `/v3/projects` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:57.351984 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-257b45c5-7d6c-468b-af86-4b3567ba675b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:49:57.352393 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-257b45c5-7d6c-468b-af86-4b3567ba675b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:57.354024 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-257b45c5-7d6c-468b-af86-4b3567ba675b None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:57.369047 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 349/706] 10.222.0.27 () {56 vars in 1148 bytes} [Fri Aug 5 19:49:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:49:57.369814 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257b45c5-7d6c-468b-af86-4b3567ba675b None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:57.474727 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbca9646-45c3-4015-b2e3-08cdce93d999 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:57.481347 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257b45c5-7d6c-468b-af86-4b3567ba675b None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:57.487423 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:57.506210 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:57.517373 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebfba6a9-e6cf-498f-992c-39573e15bbd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:57.517796 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9adb48-0064-43f1-bde3-c1fa2620d040 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:57.518028 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfba6a9-e6cf-498f-992c-39573e15bbd7 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:57.518311 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfba6a9-e6cf-498f-992c-39573e15bbd7 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:57.518588 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfba6a9-e6cf-498f-992c-39573e15bbd7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:57.518816 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9adb48-0064-43f1-bde3-c1fa2620d040 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:57.519190 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9adb48-0064-43f1-bde3-c1fa2620d040 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:57.519559 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9adb48-0064-43f1-bde3-c1fa2620d040 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:57.526104 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebfba6a9-e6cf-498f-992c-39573e15bbd7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:57.527679 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebfba6a9-e6cf-498f-992c-39573e15bbd7 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:57.530740 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b9adb48-0064-43f1-bde3-c1fa2620d040 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:57.532171 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b9adb48-0064-43f1-bde3-c1fa2620d040 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:57.539407 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-ebfba6a9-e6cf-498f-992c-39573e15bbd7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 19:49:57.539407 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:49:57.539407 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:49:57.539407 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:49:57.539407 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:49:57.539407 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:49:57.539407 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:49:57.539407 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:49:57.539407 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:49:57.539407 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:49:57.539407 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:49:57.539407 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:49:57.539407 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 05 19:49:57.539407 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 05 19:49:57.539407 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 05 19:49:57.540448 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 05 19:49:57.540448 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:49:57.540448 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:49:57.540448 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:49:57.540448 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:49:57.540448 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:49:57.540448 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:49:57.540448 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:49:57.540448 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Aug 05 19:49:57.540448 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:49:57.540448 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:49:57.540448 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:49:57.540448 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:49:57.540448 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:49:57.540448 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:49:57.540448 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:49:57.540448 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:49:57.541428 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 05 19:49:57.541428 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 05 19:49:57.541428 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 05 19:49:57.541428 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 05 19:49:57.541428 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 05 19:49:57.541428 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 05 19:49:57.541428 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 19:49:57.541428 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Aug 05 19:49:57.541428 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 358/707] 10.222.0.27 () {56 vars in 1111 bytes} [Fri Aug 5 19:49:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:49:57.542487 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebfba6a9-e6cf-498f-992c-39573e15bbd7 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:57.551537 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application [None req-2b9adb48-0064-43f1-bde3-c1fa2620d040 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:49:57.551537 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:49:57.551537 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:49:57.551537 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:49:57.551537 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:49:57.551537 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:49:57.551537 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:49:57.551537 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:49:57.551537 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:49:57.551537 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:49:57.551537 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:49:57.551537 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:49:57.551537 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 19:49:57.551537 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 19:49:57.551537 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 19:49:57.552549 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 19:49:57.552549 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:49:57.552549 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:49:57.552549 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:49:57.552549 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:49:57.552549 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:49:57.552549 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:49:57.552549 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:49:57.552549 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application with Lock( Aug 05 19:49:57.552549 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:49:57.552549 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:49:57.552549 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:49:57.552549 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:49:57.552549 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:49:57.552549 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:49:57.552549 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:49:57.552549 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:49:57.554130 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 19:49:57.554130 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 19:49:57.554130 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:49:57.554130 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Aug 05 19:49:57.554130 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 350/708] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Aug 5 19:49:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:49:57.554130 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9adb48-0064-43f1-bde3-c1fa2620d040 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:57.651249 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebfba6a9-e6cf-498f-992c-39573e15bbd7 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:57.660012 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9adb48-0064-43f1-bde3-c1fa2620d040 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:57.686519 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983d1fc0-430f-4f5f-bfe3-bcc14b121677 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:57.687370 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983d1fc0-430f-4f5f-bfe3-bcc14b121677 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:57.687660 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983d1fc0-430f-4f5f-bfe3-bcc14b121677 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:57.688350 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983d1fc0-430f-4f5f-bfe3-bcc14b121677 None admin] PATH_INFO: `/v3/services` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:57.688646 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-983d1fc0-430f-4f5f-bfe3-bcc14b121677 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:49:57.689012 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-983d1fc0-430f-4f5f-bfe3-bcc14b121677 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:57.690447 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-983d1fc0-430f-4f5f-bfe3-bcc14b121677 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:57.704539 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 359/709] 10.222.0.27 () {56 vars in 1124 bytes} [Fri Aug 5 19:49:57 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:49:57.704539 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983d1fc0-430f-4f5f-bfe3-bcc14b121677 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:57.711606 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e362e5-a6e8-455e-ad52-8eaa8bd99456 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:57.712357 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e362e5-a6e8-455e-ad52-8eaa8bd99456 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:57.712741 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e362e5-a6e8-455e-ad52-8eaa8bd99456 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:57.713480 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e362e5-a6e8-455e-ad52-8eaa8bd99456 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:57.714069 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e362e5-a6e8-455e-ad52-8eaa8bd99456 None admin] RBAC: Adding query filter params (role.id=6e2bc1ba438d4eb69923ef4659c181f3, scope.project.id=a70a82dcfc574fad90b3f5f00abbba58, user.id=6533931da2d34447881008b43372b0d5) {{(pid=80364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:49:57.714453 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e362e5-a6e8-455e-ad52-8eaa8bd99456 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:57.716284 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e362e5-a6e8-455e-ad52-8eaa8bd99456 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:57.724656 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 351/710] 10.222.0.27 () {56 vars in 1374 bytes} [Fri Aug 5 19:49:57 2022] GET /identity/v3/role_assignments?user.id=6533931da2d34447881008b43372b0d5&scope.project.id=a70a82dcfc574fad90b3f5f00abbba58&role.id=6e2bc1ba438d4eb69923ef4659c181f3 => generated 608 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:49:57.725688 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e362e5-a6e8-455e-ad52-8eaa8bd99456 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:57.775949 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:57.808857 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:57.903619 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983d1fc0-430f-4f5f-bfe3-bcc14b121677 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:57.923953 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e362e5-a6e8-455e-ad52-8eaa8bd99456 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:57.948445 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136d4a12-ccf8-4e7e-9276-788099a0e2f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:57.949104 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136d4a12-ccf8-4e7e-9276-788099a0e2f6 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:57.949386 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136d4a12-ccf8-4e7e-9276-788099a0e2f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:57.949660 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136d4a12-ccf8-4e7e-9276-788099a0e2f6 None admin] PATH_INFO: `/v3/projects` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:57.950426 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-136d4a12-ccf8-4e7e-9276-788099a0e2f6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:49:57.950733 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-136d4a12-ccf8-4e7e-9276-788099a0e2f6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:57.952817 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-136d4a12-ccf8-4e7e-9276-788099a0e2f6 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:57.971313 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 360/711] 10.222.0.27 () {56 vars in 1148 bytes} [Fri Aug 5 19:49:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:49:57.974146 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ce54ed-8846-4e12-b279-de458290353a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:57.974871 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ce54ed-8846-4e12-b279-de458290353a None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:57.975217 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ce54ed-8846-4e12-b279-de458290353a None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:57.975556 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ce54ed-8846-4e12-b279-de458290353a None admin] PATH_INFO: `/v3/domains` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:57.976320 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ce54ed-8846-4e12-b279-de458290353a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:49:57.980801 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ce54ed-8846-4e12-b279-de458290353a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:57.982925 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ce54ed-8846-4e12-b279-de458290353a None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:57.983991 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136d4a12-ccf8-4e7e-9276-788099a0e2f6 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:57.999031 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 352/712] 10.222.0.27 () {56 vars in 1109 bytes} [Fri Aug 5 19:49:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:49:58.004735 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ce54ed-8846-4e12-b279-de458290353a None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:58.094129 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:58.102746 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136d4a12-ccf8-4e7e-9276-788099a0e2f6 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:58.125199 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:58.131315 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ce54ed-8846-4e12-b279-de458290353a None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:58.146382 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5321f631-5867-47fb-8b2e-51992395b4a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:58.147116 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5321f631-5867-47fb-8b2e-51992395b4a6 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:58.147702 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5321f631-5867-47fb-8b2e-51992395b4a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:58.148039 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5321f631-5867-47fb-8b2e-51992395b4a6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:58.150154 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5321f631-5867-47fb-8b2e-51992395b4a6 None admin] RBAC: Adding query filter params (role.id=c173688a960b46e5be65f0ccd6fbd734, scope.project.id=a70a82dcfc574fad90b3f5f00abbba58, user.id=11001a2ad96f4ec8a35aab2c64376875) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:49:58.150466 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5321f631-5867-47fb-8b2e-51992395b4a6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:58.153341 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5321f631-5867-47fb-8b2e-51992395b4a6 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:58.164532 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 361/713] 10.222.0.27 () {56 vars in 1374 bytes} [Fri Aug 5 19:49:57 2022] GET /identity/v3/role_assignments?user.id=11001a2ad96f4ec8a35aab2c64376875&scope.project.id=a70a82dcfc574fad90b3f5f00abbba58&role.id=c173688a960b46e5be65f0ccd6fbd734 => generated 259 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:49:58.164532 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e487edb2-6f22-4676-b356-04fe15d88863 None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:58.164532 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e487edb2-6f22-4676-b356-04fe15d88863 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:58.164532 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e487edb2-6f22-4676-b356-04fe15d88863 None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:58.164532 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 362/714] 10.222.0.27 () {54 vars in 927 bytes} [Fri Aug 5 19:49:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:49:58.168937 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66443c94-d416-4579-a3c9-820b7cbc18e2 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:58.169218 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66443c94-d416-4579-a3c9-820b7cbc18e2 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:58.169488 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66443c94-d416-4579-a3c9-820b7cbc18e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:58.176913 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3790598-fcd9-4f3e-9a96-d53933fdd4a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:58.178602 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3790598-fcd9-4f3e-9a96-d53933fdd4a4 None admin] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:58.178884 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3790598-fcd9-4f3e-9a96-d53933fdd4a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:58.179158 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3790598-fcd9-4f3e-9a96-d53933fdd4a4 None admin] PATH_INFO: `/v3/users` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:58.179853 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3790598-fcd9-4f3e-9a96-d53933fdd4a4 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:58.181430 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3790598-fcd9-4f3e-9a96-d53933fdd4a4 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:58.189927 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-66443c94-d416-4579-a3c9-820b7cbc18e2 None None] MFA Rules not processed for user `03d1834a64cd4a43829827cb99a72747`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:49:58.191128 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-66443c94-d416-4579-a3c9-820b7cbc18e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:49:58.213067 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 353/715] 10.222.0.27 () {60 vars in 1132 bytes} [Fri Aug 5 19:49:58 2022] POST /identity/v3/users => generated 254 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 19:49:58.284351 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 363/716] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Aug 5 19:49:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 05 19:49:58.321394 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3790598-fcd9-4f3e-9a96-d53933fdd4a4 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:58.474652 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3790598-fcd9-4f3e-9a96-d53933fdd4a4 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:58.515172 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e761c2-67af-4f22-b87a-8808edd6d3d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:58.515819 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e761c2-67af-4f22-b87a-8808edd6d3d5 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:58.516101 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e761c2-67af-4f22-b87a-8808edd6d3d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:58.516374 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e761c2-67af-4f22-b87a-8808edd6d3d5 None admin] PATH_INFO: `/v3/services/network` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:58.517024 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e761c2-67af-4f22-b87a-8808edd6d3d5 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:58.518394 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e761c2-67af-4f22-b87a-8808edd6d3d5 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:58.527680 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application [None req-46e761c2-67af-4f22-b87a-8808edd6d3d5 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 05 19:49:58.527680 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:49:58.527680 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 05 19:49:58.527680 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 05 19:49:58.527680 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 05 19:49:58.527680 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 05 19:49:58.527680 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 05 19:49:58.527680 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 05 19:49:58.527680 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 05 19:49:58.527680 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Aug 05 19:49:58.527680 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 05 19:49:58.527680 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Aug 05 19:49:58.527680 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:49:58.527680 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:49:58.527680 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:49:58.527680 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:49:58.527680 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:49:58.528892 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:49:58.528892 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:49:58.528892 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:49:58.528892 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:49:58.528892 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:49:58.528892 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:49:58.528892 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 05 19:49:58.528892 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 05 19:49:58.528892 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 05 19:49:58.528892 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 05 19:49:58.528892 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:49:58.528892 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:49:58.528892 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:49:58.528892 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:49:58.528892 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:49:58.528892 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:49:58.530495 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:49:58.530495 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application with Lock( Aug 05 19:49:58.530495 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:49:58.530495 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:49:58.530495 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:49:58.530495 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:49:58.530495 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:49:58.530495 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:49:58.530495 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:49:58.530495 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:49:58.530495 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 05 19:49:58.530495 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 05 19:49:58.530495 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 05 19:49:58.530495 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Aug 05 19:49:58.530495 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 354/717] 10.222.0.27 () {56 vars in 1111 bytes} [Fri Aug 5 19:49:58 2022] GET /identity/v3/services/network => generated 88 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:49:58.535473 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66443c94-d416-4579-a3c9-820b7cbc18e2 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:58.632004 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:58.646844 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66443c94-d416-4579-a3c9-820b7cbc18e2 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:58.688461 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ade1337-ee7d-4743-a25b-a1400c1d5a43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:58.689112 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ade1337-ee7d-4743-a25b-a1400c1d5a43 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:58.689394 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ade1337-ee7d-4743-a25b-a1400c1d5a43 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:58.689666 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ade1337-ee7d-4743-a25b-a1400c1d5a43 None admin] PATH_INFO: `/v3/services` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:58.690356 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ade1337-ee7d-4743-a25b-a1400c1d5a43 None admin] RBAC: Adding query filter params (name=network) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:49:58.690662 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ade1337-ee7d-4743-a25b-a1400c1d5a43 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:58.692031 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ade1337-ee7d-4743-a25b-a1400c1d5a43 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:58.703088 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 364/718] 10.222.0.27 () {56 vars in 1112 bytes} [Fri Aug 5 19:49:58 2022] GET /identity/v3/services?name=network => generated 124 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:49:58.708299 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e761c2-67af-4f22-b87a-8808edd6d3d5 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:49:58.759628 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:49:58.864148 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e761c2-67af-4f22-b87a-8808edd6d3d5 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:49:58.898910 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a62464-1f4d-48d4-a737-5267b6de6426 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:49:58.900128 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a62464-1f4d-48d4-a737-5267b6de6426 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:49:58.900670 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a62464-1f4d-48d4-a737-5267b6de6426 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:49:58.901151 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a62464-1f4d-48d4-a737-5267b6de6426 None admin] PATH_INFO: `/v3/services` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:49:58.902045 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a62464-1f4d-48d4-a737-5267b6de6426 None admin] RBAC: Adding query filter params (type=network) {{(pid=80364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:49:58.902618 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a62464-1f4d-48d4-a737-5267b6de6426 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:49:58.904288 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a62464-1f4d-48d4-a737-5267b6de6426 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:49:58.910752 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 355/719] 10.222.0.27 () {56 vars in 1112 bytes} [Fri Aug 5 19:49:58 2022] GET /identity/v3/services?type=network => generated 124 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:50:00.303985 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a31b21-476c-4428-b124-1407e67d6559 None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:00.304553 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a31b21-476c-4428-b124-1407e67d6559 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:00.304553 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a31b21-476c-4428-b124-1407e67d6559 None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:00.306193 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 365/720] 10.222.0.27 () {54 vars in 927 bytes} [Fri Aug 5 19:50:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:50:00.315442 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ca428a-a39c-4636-9a13-c864d6cf8635 None None] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:00.315890 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ca428a-a39c-4636-9a13-c864d6cf8635 None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:00.316075 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ca428a-a39c-4636-9a13-c864d6cf8635 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:00.341569 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.auth.core [None req-c2ca428a-a39c-4636-9a13-c864d6cf8635 None None] MFA Rules not processed for user `03d1834a64cd4a43829827cb99a72747`. Rule list: `[]` (Enabled: `True`). {{(pid=80364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:50:00.342790 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-c2ca428a-a39c-4636-9a13-c864d6cf8635 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:50:00.440434 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 356/721] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Aug 5 19:50:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 19:50:00.472327 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a31b21-476c-4428-b124-1407e67d6559 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:00.574133 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97efbc91-a028-4f68-aeae-5a66c27cc09e None None] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:00.574435 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97efbc91-a028-4f68-aeae-5a66c27cc09e None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:00.574629 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97efbc91-a028-4f68-aeae-5a66c27cc09e None None] PATH_INFO: `/` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:00.576235 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 357/722] 10.222.0.27 () {54 vars in 927 bytes} [Fri Aug 5 19:50:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:50:00.581352 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903cb115-d0c9-4c71-9c24-b874dfdbcdd5 None None] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:00.581641 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903cb115-d0c9-4c71-9c24-b874dfdbcdd5 None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:00.581918 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903cb115-d0c9-4c71-9c24-b874dfdbcdd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:00.599981 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.auth.core [None req-903cb115-d0c9-4c71-9c24-b874dfdbcdd5 None None] MFA Rules not processed for user `03d1834a64cd4a43829827cb99a72747`. Rule list: `[]` (Enabled: `True`). {{(pid=80364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:50:00.601295 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-903cb115-d0c9-4c71-9c24-b874dfdbcdd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:50:00.602194 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a31b21-476c-4428-b124-1407e67d6559 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:00.613626 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:00.678243 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 358/723] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Aug 5 19:50:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 05 19:50:00.702950 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903cb115-d0c9-4c71-9c24-b874dfdbcdd5 None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:00.703174 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d3a375-36c6-4cb2-84c7-e9a8a3a31e82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:00.703623 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d3a375-36c6-4cb2-84c7-e9a8a3a31e82 None admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:00.703904 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d3a375-36c6-4cb2-84c7-e9a8a3a31e82 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:00.704179 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d3a375-36c6-4cb2-84c7-e9a8a3a31e82 None admin] PATH_INFO: `/v3/services` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:00.704927 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d3a375-36c6-4cb2-84c7-e9a8a3a31e82 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:00.706302 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d3a375-36c6-4cb2-84c7-e9a8a3a31e82 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:00.750471 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 366/724] 10.222.0.27 () {60 vars in 1141 bytes} [Fri Aug 5 19:50:00 2022] POST /identity/v3/services => generated 251 bytes in 278 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 19:50:00.818410 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2a1e9f-9603-468a-ad4d-f06f2c719eb5 None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:00.818744 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2a1e9f-9603-468a-ad4d-f06f2c719eb5 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:00.818972 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2a1e9f-9603-468a-ad4d-f06f2c719eb5 None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:00.821408 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903cb115-d0c9-4c71-9c24-b874dfdbcdd5 None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:00.824560 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 367/725] 10.222.0.27 () {54 vars in 927 bytes} [Fri Aug 5 19:50:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 05 19:50:00.834151 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2208e8f4-3ea9-40c6-b956-ef7c622281c3 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:00.834487 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2208e8f4-3ea9-40c6-b956-ef7c622281c3 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:00.834706 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2208e8f4-3ea9-40c6-b956-ef7c622281c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:00.874390 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-2208e8f4-3ea9-40c6-b956-ef7c622281c3 None None] MFA Rules not processed for user `03d1834a64cd4a43829827cb99a72747`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:50:00.875673 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-2208e8f4-3ea9-40c6-b956-ef7c622281c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:50:00.892908 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:00.996987 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 368/726] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Aug 5 19:50:00 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 05 19:50:01.000371 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576bcf78-6761-427a-8f90-0cc984590026 None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:01.000727 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576bcf78-6761-427a-8f90-0cc984590026 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:01.001036 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576bcf78-6761-427a-8f90-0cc984590026 None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:01.003094 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 369/727] 10.222.0.27 () {54 vars in 927 bytes} [Fri Aug 5 19:50:00 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:50:01.009444 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860dc78d-40e0-4bd8-8d00-71ebb5f57391 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:01.009853 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c219d2e-ce9d-41a3-9000-325fb3fd3eda None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:01.010176 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c219d2e-ce9d-41a3-9000-325fb3fd3eda None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:01.010176 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c219d2e-ce9d-41a3-9000-325fb3fd3eda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:01.011097 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860dc78d-40e0-4bd8-8d00-71ebb5f57391 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:01.011634 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860dc78d-40e0-4bd8-8d00-71ebb5f57391 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:01.011999 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860dc78d-40e0-4bd8-8d00-71ebb5f57391 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:01.025190 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-860dc78d-40e0-4bd8-8d00-71ebb5f57391 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 19:50:01.030947 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application [None req-860dc78d-40e0-4bd8-8d00-71ebb5f57391 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 19:50:01.030947 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:50:01.030947 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:50:01.030947 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:50:01.030947 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:50:01.030947 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:50:01.030947 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:50:01.030947 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:50:01.030947 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:50:01.030947 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:50:01.030947 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:50:01.030947 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:50:01.030947 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 19:50:01.030947 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 19:50:01.030947 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 19:50:01.030947 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 19:50:01.033186 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 19:50:01.033186 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 19:50:01.033186 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 19:50:01.033186 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 19:50:01.033186 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:50:01.033186 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:50:01.033186 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:50:01.033186 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:50:01.033186 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:50:01.033186 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:50:01.033186 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:50:01.033186 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application with Lock( Aug 05 19:50:01.033186 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:50:01.033186 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:50:01.033186 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:50:01.033186 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:50:01.035290 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:50:01.035290 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:50:01.035290 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:50:01.035290 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:50:01.035290 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 19:50:01.035290 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 19:50:01.035290 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 19:50:01.035290 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 19:50:01.035290 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 19:50:01.035290 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 19:50:01.035290 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 19:50:01.035290 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Aug 05 19:50:01.036870 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-8c219d2e-ce9d-41a3-9000-325fb3fd3eda None None] MFA Rules not processed for user `03d1834a64cd4a43829827cb99a72747`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:50:01.037020 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 359/728] 10.222.0.27 () {56 vars in 1096 bytes} [Fri Aug 5 19:50:00 2022] GET /identity/v3/roles/admin => generated 83 bytes in 330 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:50:01.037020 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860dc78d-40e0-4bd8-8d00-71ebb5f57391 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:01.037298 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-8c219d2e-ce9d-41a3-9000-325fb3fd3eda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:50:01.042666 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:01.112883 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 370/729] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Aug 5 19:50:01 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 19:50:01.113661 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c219d2e-ce9d-41a3-9000-325fb3fd3eda None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:01.159611 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860dc78d-40e0-4bd8-8d00-71ebb5f57391 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:01.184923 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:01.198531 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f791212-44bb-4aed-acf3-8f62bd8e3f22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:01.199178 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f791212-44bb-4aed-acf3-8f62bd8e3f22 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:01.199460 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f791212-44bb-4aed-acf3-8f62bd8e3f22 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:01.199736 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f791212-44bb-4aed-acf3-8f62bd8e3f22 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:01.211031 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f791212-44bb-4aed-acf3-8f62bd8e3f22 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 19:50:01.216357 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application [None req-5f791212-44bb-4aed-acf3-8f62bd8e3f22 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 19:50:01.216357 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:50:01.216357 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:50:01.216357 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:50:01.216357 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:50:01.216357 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:50:01.216357 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:50:01.216357 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:50:01.216357 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:50:01.216357 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:50:01.216357 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:50:01.216357 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:50:01.216357 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 19:50:01.216357 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 19:50:01.216357 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 19:50:01.216357 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 19:50:01.218577 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 19:50:01.218577 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 19:50:01.218577 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 19:50:01.218577 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 19:50:01.218577 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:50:01.218577 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:50:01.218577 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:50:01.218577 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:50:01.218577 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:50:01.218577 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:50:01.218577 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:50:01.218577 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application with Lock( Aug 05 19:50:01.218577 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:50:01.218577 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:50:01.218577 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:50:01.218577 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:50:01.220823 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:50:01.220823 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:50:01.220823 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:50:01.220823 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:50:01.220823 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 19:50:01.220823 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 19:50:01.220823 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 19:50:01.220823 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 19:50:01.220823 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 19:50:01.220823 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 19:50:01.220823 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 19:50:01.220823 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Aug 05 19:50:01.222396 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c219d2e-ce9d-41a3-9000-325fb3fd3eda None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:01.222561 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 360/730] 10.222.0.27 () {56 vars in 1096 bytes} [Fri Aug 5 19:50:01 2022] GET /identity/v3/roles/admin => generated 83 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:50:01.222561 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f791212-44bb-4aed-acf3-8f62bd8e3f22 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:01.253654 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e99007-1619-40ac-9379-5f2eff38705b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:01.254281 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e99007-1619-40ac-9379-5f2eff38705b None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:01.254578 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e99007-1619-40ac-9379-5f2eff38705b None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:01.254825 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e99007-1619-40ac-9379-5f2eff38705b None admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:01.255545 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e99007-1619-40ac-9379-5f2eff38705b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:50:01.255842 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e99007-1619-40ac-9379-5f2eff38705b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:01.257235 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e99007-1619-40ac-9379-5f2eff38705b None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:01.266433 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 371/731] 10.222.0.27 () {56 vars in 1099 bytes} [Fri Aug 5 19:50:01 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:50:01.267260 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e99007-1619-40ac-9379-5f2eff38705b None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:01.286203 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:01.333146 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f791212-44bb-4aed-acf3-8f62bd8e3f22 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:01.378181 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13cf95f-64ea-4ade-9663-910f6e9de618 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:01.378801 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13cf95f-64ea-4ade-9663-910f6e9de618 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:01.379090 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13cf95f-64ea-4ade-9663-910f6e9de618 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:01.379360 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13cf95f-64ea-4ade-9663-910f6e9de618 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:01.382181 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e99007-1619-40ac-9379-5f2eff38705b None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:01.391292 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13cf95f-64ea-4ade-9663-910f6e9de618 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 19:50:01.396877 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application [None req-e13cf95f-64ea-4ade-9663-910f6e9de618 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 05 19:50:01.396877 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:50:01.396877 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:50:01.396877 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:50:01.396877 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:50:01.396877 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:50:01.396877 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:50:01.396877 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:50:01.396877 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:50:01.396877 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:50:01.396877 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:50:01.396877 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:50:01.396877 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 19:50:01.396877 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 19:50:01.396877 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 19:50:01.396877 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 19:50:01.397809 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 19:50:01.397809 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 19:50:01.397809 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 19:50:01.397809 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 19:50:01.397809 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:50:01.397809 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:50:01.397809 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:50:01.397809 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:50:01.397809 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:50:01.397809 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:50:01.397809 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:50:01.397809 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application with Lock( Aug 05 19:50:01.397809 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:50:01.397809 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:50:01.397809 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:50:01.397809 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:50:01.398667 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:50:01.398667 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:50:01.398667 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:50:01.398667 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:50:01.398667 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 19:50:01.398667 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 19:50:01.398667 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 19:50:01.398667 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 19:50:01.398667 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 19:50:01.398667 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 19:50:01.398667 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 05 19:50:01.398667 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Aug 05 19:50:01.398667 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 361/732] 10.222.0.27 () {56 vars in 1102 bytes} [Fri Aug 5 19:50:01 2022] GET /identity/v3/roles/service => generated 85 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:50:01.399354 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13cf95f-64ea-4ade-9663-910f6e9de618 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:01.419441 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:01.427012 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266e06ff-5cb3-42ce-9a68-0a26afa5fdd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:01.427622 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266e06ff-5cb3-42ce-9a68-0a26afa5fdd0 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:01.427900 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266e06ff-5cb3-42ce-9a68-0a26afa5fdd0 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:01.428168 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266e06ff-5cb3-42ce-9a68-0a26afa5fdd0 None admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:01.429027 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-266e06ff-5cb3-42ce-9a68-0a26afa5fdd0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:50:01.429517 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-266e06ff-5cb3-42ce-9a68-0a26afa5fdd0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:01.430684 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-266e06ff-5cb3-42ce-9a68-0a26afa5fdd0 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:01.439562 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 372/733] 10.222.0.27 () {56 vars in 1099 bytes} [Fri Aug 5 19:50:01 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:50:01.440336 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c96328-3e44-4d24-95c7-b22e8cdb4a46 None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:01.440647 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c96328-3e44-4d24-95c7-b22e8cdb4a46 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:01.440918 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c96328-3e44-4d24-95c7-b22e8cdb4a46 None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:01.444615 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 373/734] 10.222.0.27 () {54 vars in 927 bytes} [Fri Aug 5 19:50:01 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:50:01.444615 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c96328-3e44-4d24-95c7-b22e8cdb4a46 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:01.517388 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13cf95f-64ea-4ade-9663-910f6e9de618 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:01.548053 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:01.561728 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe12893-d989-42cb-b365-2fd402fc7e0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:01.562376 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe12893-d989-42cb-b365-2fd402fc7e0a None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:01.562661 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe12893-d989-42cb-b365-2fd402fc7e0a None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:01.562935 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe12893-d989-42cb-b365-2fd402fc7e0a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:01.573360 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe12893-d989-42cb-b365-2fd402fc7e0a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:01.573565 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c96328-3e44-4d24-95c7-b22e8cdb4a46 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:01.574825 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe12893-d989-42cb-b365-2fd402fc7e0a None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:01.590410 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application [None req-1fe12893-d989-42cb-b365-2fd402fc7e0a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:50:01.590410 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:50:01.590410 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:50:01.590410 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:50:01.590410 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:50:01.590410 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:50:01.590410 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:50:01.590410 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:50:01.590410 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:50:01.590410 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:50:01.590410 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:50:01.590410 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:50:01.590410 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 19:50:01.590410 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 19:50:01.590410 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 19:50:01.591337 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 19:50:01.591337 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:50:01.591337 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:50:01.591337 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:50:01.591337 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:50:01.591337 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:50:01.591337 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:50:01.591337 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:50:01.591337 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application with Lock( Aug 05 19:50:01.591337 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:50:01.591337 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:50:01.591337 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:50:01.591337 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:50:01.591337 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:50:01.591337 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:50:01.591337 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:50:01.591337 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:50:01.592310 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 19:50:01.592310 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 19:50:01.592310 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:50:01.592310 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Aug 05 19:50:01.592310 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 362/735] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Aug 5 19:50:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:50:01.593038 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe12893-d989-42cb-b365-2fd402fc7e0a None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:01.616109 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09e4867-597d-4c8d-8604-a8775092d251 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:01.616771 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09e4867-597d-4c8d-8604-a8775092d251 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:01.617056 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09e4867-597d-4c8d-8604-a8775092d251 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:01.617439 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09e4867-597d-4c8d-8604-a8775092d251 None admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:01.618208 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09e4867-597d-4c8d-8604-a8775092d251 None admin] RBAC: Adding query filter params (name=service) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:50:01.618461 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09e4867-597d-4c8d-8604-a8775092d251 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:01.619822 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09e4867-597d-4c8d-8604-a8775092d251 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:01.629173 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 374/736] 10.222.0.27 () {56 vars in 1103 bytes} [Fri Aug 5 19:50:01 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:50:01.630369 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf92c5e-90ad-46c0-adee-40472de76ea5 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:01.630656 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf92c5e-90ad-46c0-adee-40472de76ea5 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:01.630927 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf92c5e-90ad-46c0-adee-40472de76ea5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:01.650046 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-5cf92c5e-90ad-46c0-adee-40472de76ea5 None None] MFA Rules not processed for user `03d1834a64cd4a43829827cb99a72747`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:50:01.651319 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-5cf92c5e-90ad-46c0-adee-40472de76ea5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:50:01.701409 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:01.702458 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe12893-d989-42cb-b365-2fd402fc7e0a None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:01.720736 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 375/737] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Aug 5 19:50:01 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 05 19:50:01.729712 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf92c5e-90ad-46c0-adee-40472de76ea5 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:01.739609 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1df4e7-eeca-437c-afe6-81633925edf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:01.740439 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1df4e7-eeca-437c-afe6-81633925edf2 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:01.741038 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1df4e7-eeca-437c-afe6-81633925edf2 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:01.741316 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1df4e7-eeca-437c-afe6-81633925edf2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:01.753154 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1df4e7-eeca-437c-afe6-81633925edf2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:01.754598 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1df4e7-eeca-437c-afe6-81633925edf2 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:01.768987 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application [None req-ef1df4e7-eeca-437c-afe6-81633925edf2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:50:01.768987 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:50:01.768987 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:50:01.768987 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:50:01.768987 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:50:01.768987 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:50:01.768987 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:50:01.768987 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:50:01.768987 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:50:01.768987 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:50:01.768987 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:50:01.768987 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:50:01.768987 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 19:50:01.768987 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 19:50:01.768987 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 19:50:01.769898 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 19:50:01.769898 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:50:01.769898 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:50:01.769898 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:50:01.769898 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:50:01.769898 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:50:01.769898 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:50:01.769898 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:50:01.769898 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application with Lock( Aug 05 19:50:01.769898 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:50:01.769898 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:50:01.769898 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:50:01.769898 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:50:01.769898 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:50:01.769898 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:50:01.769898 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:50:01.769898 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:50:01.770906 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 19:50:01.770906 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 19:50:01.770906 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:50:01.770906 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Aug 05 19:50:01.770906 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 363/738] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Aug 5 19:50:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:50:01.771229 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1df4e7-eeca-437c-afe6-81633925edf2 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:01.800318 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:01.844780 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf92c5e-90ad-46c0-adee-40472de76ea5 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:01.877437 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1df4e7-eeca-437c-afe6-81633925edf2 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:01.880991 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b47708c-c115-499b-aff9-994c73b38c9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:01.881614 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b47708c-c115-499b-aff9-994c73b38c9e None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:01.881896 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b47708c-c115-499b-aff9-994c73b38c9e None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:01.882171 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b47708c-c115-499b-aff9-994c73b38c9e None admin] PATH_INFO: `/v3/domains` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:01.882920 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b47708c-c115-499b-aff9-994c73b38c9e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:50:01.883252 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b47708c-c115-499b-aff9-994c73b38c9e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:01.884759 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b47708c-c115-499b-aff9-994c73b38c9e None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:01.898340 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 376/739] 10.222.0.27 () {56 vars in 1109 bytes} [Fri Aug 5 19:50:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:50:01.899357 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b47708c-c115-499b-aff9-994c73b38c9e None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:01.916245 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88b45a3-8c2d-469f-9146-973db4f42213 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:01.917032 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88b45a3-8c2d-469f-9146-973db4f42213 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:01.917376 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88b45a3-8c2d-469f-9146-973db4f42213 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:01.917716 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88b45a3-8c2d-469f-9146-973db4f42213 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:01.930308 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c88b45a3-8c2d-469f-9146-973db4f42213 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:01.931802 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c88b45a3-8c2d-469f-9146-973db4f42213 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:01.946914 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:01.950601 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application [None req-c88b45a3-8c2d-469f-9146-973db4f42213 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:50:01.950601 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:50:01.950601 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:50:01.950601 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:50:01.950601 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:50:01.950601 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:50:01.950601 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:50:01.950601 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:50:01.950601 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:50:01.950601 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:50:01.950601 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:50:01.950601 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:50:01.950601 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 19:50:01.950601 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 19:50:01.950601 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 19:50:01.951718 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 19:50:01.951718 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:50:01.951718 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:50:01.951718 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:50:01.951718 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:50:01.951718 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:50:01.951718 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:50:01.951718 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:50:01.951718 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application with Lock( Aug 05 19:50:01.951718 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:50:01.951718 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:50:01.951718 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:50:01.951718 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:50:01.951718 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:50:01.951718 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:50:01.951718 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:50:01.951718 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:50:01.952848 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 19:50:01.952848 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 19:50:01.952848 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:50:01.952848 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Aug 05 19:50:01.956406 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 364/740] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Aug 5 19:50:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:50:01.957267 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88b45a3-8c2d-469f-9146-973db4f42213 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:02.008486 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b47708c-c115-499b-aff9-994c73b38c9e None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:02.045667 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:02.046957 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18dfb311-5f8c-4c8d-8ad8-0d8f4269da3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:02.047698 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18dfb311-5f8c-4c8d-8ad8-0d8f4269da3a None admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:02.047894 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18dfb311-5f8c-4c8d-8ad8-0d8f4269da3a None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:02.048123 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18dfb311-5f8c-4c8d-8ad8-0d8f4269da3a None admin] PATH_INFO: `/v3/services` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:02.048880 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18dfb311-5f8c-4c8d-8ad8-0d8f4269da3a None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:02.050297 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18dfb311-5f8c-4c8d-8ad8-0d8f4269da3a None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:02.072533 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 377/741] 10.222.0.27 () {60 vars in 1141 bytes} [Fri Aug 5 19:50:01 2022] POST /identity/v3/services => generated 240 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 19:50:02.072533 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18dfb311-5f8c-4c8d-8ad8-0d8f4269da3a None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:02.078248 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88b45a3-8c2d-469f-9146-973db4f42213 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:02.140407 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade81d7e-d23d-4959-a355-9353e9df10e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:02.141027 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade81d7e-d23d-4959-a355-9353e9df10e3 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:02.141317 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade81d7e-d23d-4959-a355-9353e9df10e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:02.141528 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade81d7e-d23d-4959-a355-9353e9df10e3 None admin] PATH_INFO: `/v3/domains` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:02.142239 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade81d7e-d23d-4959-a355-9353e9df10e3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:50:02.142527 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade81d7e-d23d-4959-a355-9353e9df10e3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:02.144100 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade81d7e-d23d-4959-a355-9353e9df10e3 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:02.158602 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 365/742] 10.222.0.27 () {56 vars in 1109 bytes} [Fri Aug 5 19:50:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:50:02.159330 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade81d7e-d23d-4959-a355-9353e9df10e3 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:02.225098 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18dfb311-5f8c-4c8d-8ad8-0d8f4269da3a None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:02.228286 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:02.264887 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a3d011-eb2d-47c0-9d6c-4e858b6c9385 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:02.265771 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a3d011-eb2d-47c0-9d6c-4e858b6c9385 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:02.266187 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a3d011-eb2d-47c0-9d6c-4e858b6c9385 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:02.266607 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a3d011-eb2d-47c0-9d6c-4e858b6c9385 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:02.271664 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade81d7e-d23d-4959-a355-9353e9df10e3 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:02.275386 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a3d011-eb2d-47c0-9d6c-4e858b6c9385 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:02.276947 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a3d011-eb2d-47c0-9d6c-4e858b6c9385 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:02.290154 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-60a3d011-eb2d-47c0-9d6c-4e858b6c9385 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 05 19:50:02.290154 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:50:02.290154 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:50:02.290154 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:50:02.290154 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:50:02.290154 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:50:02.290154 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:50:02.290154 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:50:02.290154 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:50:02.290154 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:50:02.290154 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:50:02.290154 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:50:02.290154 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 05 19:50:02.290154 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 05 19:50:02.290154 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 05 19:50:02.290154 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 05 19:50:02.291141 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:50:02.291141 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:50:02.291141 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 05 19:50:02.291141 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 19:50:02.291141 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 05 19:50:02.291141 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 19:50:02.291141 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:50:02.291141 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:50:02.291141 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:50:02.291141 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:50:02.291141 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:50:02.291141 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Aug 05 19:50:02.291141 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:50:02.291141 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:50:02.291141 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:50:02.291141 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:50:02.291141 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:50:02.291141 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:50:02.292076 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:50:02.292076 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:50:02.292076 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 05 19:50:02.292076 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 05 19:50:02.292076 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 05 19:50:02.292076 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 05 19:50:02.292076 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 05 19:50:02.292076 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 05 19:50:02.292076 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 05 19:50:02.292076 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Aug 05 19:50:02.292076 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 378/743] 10.222.0.27 () {56 vars in 1096 bytes} [Fri Aug 5 19:50:02 2022] GET /identity/v3/users/swift => generated 83 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:50:02.292076 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a3d011-eb2d-47c0-9d6c-4e858b6c9385 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:02.320220 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c3b304-4b13-47f1-8438-fc756925b458 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:02.321665 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c3b304-4b13-47f1-8438-fc756925b458 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:02.321947 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c3b304-4b13-47f1-8438-fc756925b458 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:02.322223 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c3b304-4b13-47f1-8438-fc756925b458 None admin] PATH_INFO: `/v3/domains` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:02.322938 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c3b304-4b13-47f1-8438-fc756925b458 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:50:02.323242 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c3b304-4b13-47f1-8438-fc756925b458 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:02.327075 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c3b304-4b13-47f1-8438-fc756925b458 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:02.333134 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:02.351001 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 366/744] 10.222.0.27 () {56 vars in 1109 bytes} [Fri Aug 5 19:50:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:50:02.352208 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c3b304-4b13-47f1-8438-fc756925b458 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:02.424072 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a3d011-eb2d-47c0-9d6c-4e858b6c9385 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:02.463024 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a5fac3-9486-4117-8872-aae2c54a856a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:02.463792 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a5fac3-9486-4117-8872-aae2c54a856a None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:02.464073 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a5fac3-9486-4117-8872-aae2c54a856a None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:02.464349 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a5fac3-9486-4117-8872-aae2c54a856a None admin] PATH_INFO: `/v3/users/nova` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:02.473761 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a5fac3-9486-4117-8872-aae2c54a856a None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:02.475198 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a5fac3-9486-4117-8872-aae2c54a856a None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:02.485755 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c3b304-4b13-47f1-8438-fc756925b458 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:02.488209 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-b0a5fac3-9486-4117-8872-aae2c54a856a None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 05 19:50:02.488209 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:50:02.488209 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:50:02.488209 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:50:02.488209 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:50:02.488209 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:50:02.488209 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:50:02.488209 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:50:02.488209 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:50:02.488209 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:50:02.488209 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:50:02.488209 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:50:02.488209 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 05 19:50:02.488209 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 05 19:50:02.488209 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 05 19:50:02.488209 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 05 19:50:02.489345 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:50:02.489345 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:50:02.489345 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 05 19:50:02.489345 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 19:50:02.489345 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 05 19:50:02.489345 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 19:50:02.489345 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:50:02.489345 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:50:02.489345 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:50:02.489345 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:50:02.489345 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:50:02.489345 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Aug 05 19:50:02.489345 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:50:02.489345 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:50:02.489345 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:50:02.489345 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:50:02.489345 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:50:02.489345 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:50:02.490511 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:50:02.490511 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:50:02.490511 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 05 19:50:02.490511 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 05 19:50:02.490511 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 05 19:50:02.490511 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 05 19:50:02.490511 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 05 19:50:02.490511 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 05 19:50:02.490511 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 05 19:50:02.490511 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Aug 05 19:50:02.490511 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 379/745] 10.222.0.27 () {56 vars in 1093 bytes} [Fri Aug 5 19:50:02 2022] GET /identity/v3/users/nova => generated 82 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:50:02.491174 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a5fac3-9486-4117-8872-aae2c54a856a None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:02.519572 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:02.524900 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e53b86-0422-4d10-9c0b-5b7e1a06adc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:02.525718 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e53b86-0422-4d10-9c0b-5b7e1a06adc4 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:02.526145 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e53b86-0422-4d10-9c0b-5b7e1a06adc4 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:02.526564 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e53b86-0422-4d10-9c0b-5b7e1a06adc4 None admin] PATH_INFO: `/v3/users` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:02.527509 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e53b86-0422-4d10-9c0b-5b7e1a06adc4 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:50:02.528057 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e53b86-0422-4d10-9c0b-5b7e1a06adc4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:02.529936 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e53b86-0422-4d10-9c0b-5b7e1a06adc4 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:02.548955 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 367/746] 10.222.0.27 () {56 vars in 1135 bytes} [Fri Aug 5 19:50:02 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:50:02.550096 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e53b86-0422-4d10-9c0b-5b7e1a06adc4 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:02.605515 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:02.612988 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a5fac3-9486-4117-8872-aae2c54a856a None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:02.651071 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14975ea6-151a-4610-8446-a6344acfe756 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:02.651717 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14975ea6-151a-4610-8446-a6344acfe756 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:02.651997 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14975ea6-151a-4610-8446-a6344acfe756 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:02.652272 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14975ea6-151a-4610-8446-a6344acfe756 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:02.653360 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e53b86-0422-4d10-9c0b-5b7e1a06adc4 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:02.660852 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14975ea6-151a-4610-8446-a6344acfe756 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:02.662429 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14975ea6-151a-4610-8446-a6344acfe756 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:02.675755 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-14975ea6-151a-4610-8446-a6344acfe756 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 05 19:50:02.675755 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:50:02.675755 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:50:02.675755 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:50:02.675755 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:50:02.675755 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:50:02.675755 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:50:02.675755 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:50:02.675755 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:50:02.675755 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:50:02.675755 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:50:02.675755 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:50:02.675755 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 05 19:50:02.675755 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 05 19:50:02.675755 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 05 19:50:02.675755 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 05 19:50:02.676958 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:50:02.676958 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:50:02.676958 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 05 19:50:02.676958 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 19:50:02.676958 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 05 19:50:02.676958 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 19:50:02.676958 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:50:02.676958 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:50:02.676958 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:50:02.676958 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:50:02.676958 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:50:02.676958 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Aug 05 19:50:02.676958 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:50:02.676958 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:50:02.676958 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:50:02.676958 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:50:02.676958 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:50:02.676958 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:50:02.678087 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:50:02.678087 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:50:02.678087 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 05 19:50:02.678087 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 05 19:50:02.678087 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 05 19:50:02.678087 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 05 19:50:02.678087 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 05 19:50:02.678087 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 05 19:50:02.678087 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 05 19:50:02.678087 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Aug 05 19:50:02.682998 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 380/747] 10.222.0.27 () {56 vars in 1117 bytes} [Fri Aug 5 19:50:02 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:50:02.682998 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14975ea6-151a-4610-8446-a6344acfe756 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:02.700133 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb91bb05-50e4-45ef-bf7b-26c733734e94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:02.701298 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb91bb05-50e4-45ef-bf7b-26c733734e94 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:02.702319 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb91bb05-50e4-45ef-bf7b-26c733734e94 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:02.702767 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb91bb05-50e4-45ef-bf7b-26c733734e94 None admin] PATH_INFO: `/v3/users` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:02.704020 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb91bb05-50e4-45ef-bf7b-26c733734e94 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:50:02.704700 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb91bb05-50e4-45ef-bf7b-26c733734e94 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:02.706330 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb91bb05-50e4-45ef-bf7b-26c733734e94 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:02.735819 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 368/748] 10.222.0.27 () {56 vars in 1133 bytes} [Fri Aug 5 19:50:02 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:50:02.735819 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb91bb05-50e4-45ef-bf7b-26c733734e94 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:02.786824 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:02.804813 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14975ea6-151a-4610-8446-a6344acfe756 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:02.841013 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667b99ec-0f7d-4d25-bef3-b6691afa62c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:02.841700 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667b99ec-0f7d-4d25-bef3-b6691afa62c4 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:02.841981 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667b99ec-0f7d-4d25-bef3-b6691afa62c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:02.842463 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667b99ec-0f7d-4d25-bef3-b6691afa62c4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:02.845235 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb91bb05-50e4-45ef-bf7b-26c733734e94 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:02.859673 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:02.882721 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d721ec84-fa00-4ee7-9ffc-321bb5cadc28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:02.883639 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d721ec84-fa00-4ee7-9ffc-321bb5cadc28 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:02.884085 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d721ec84-fa00-4ee7-9ffc-321bb5cadc28 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:02.884518 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d721ec84-fa00-4ee7-9ffc-321bb5cadc28 None admin] PATH_INFO: `/v3/users` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:02.885451 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d721ec84-fa00-4ee7-9ffc-321bb5cadc28 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:50:02.885900 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d721ec84-fa00-4ee7-9ffc-321bb5cadc28 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:02.887459 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d721ec84-fa00-4ee7-9ffc-321bb5cadc28 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:02.906408 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 369/749] 10.222.0.27 () {56 vars in 1149 bytes} [Fri Aug 5 19:50:02 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:50:02.907452 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d721ec84-fa00-4ee7-9ffc-321bb5cadc28 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:02.920707 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-667b99ec-0f7d-4d25-bef3-b6691afa62c4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:02.922164 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-667b99ec-0f7d-4d25-bef3-b6691afa62c4 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:03.006872 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:03.016256 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d721ec84-fa00-4ee7-9ffc-321bb5cadc28 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:03.025215 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 381/750] 10.222.0.27 () {58 vars in 1282 bytes} [Fri Aug 5 19:50:02 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 05 19:50:03.025547 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da55cec9-91f3-400d-b49f-6acf8d51bf1f None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:03.025787 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da55cec9-91f3-400d-b49f-6acf8d51bf1f None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:03.026183 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da55cec9-91f3-400d-b49f-6acf8d51bf1f None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:03.027630 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 382/751] 10.222.0.27 () {54 vars in 927 bytes} [Fri Aug 5 19:50:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:50:03.028264 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da55cec9-91f3-400d-b49f-6acf8d51bf1f None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:03.063779 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d288a9-6987-4695-bd6d-f4056faf38ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:03.064481 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d288a9-6987-4695-bd6d-f4056faf38ef None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:03.064787 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d288a9-6987-4695-bd6d-f4056faf38ef None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:03.065056 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d288a9-6987-4695-bd6d-f4056faf38ef None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:03.076472 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d288a9-6987-4695-bd6d-f4056faf38ef None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:03.078091 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d288a9-6987-4695-bd6d-f4056faf38ef None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:03.094815 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application [None req-24d288a9-6987-4695-bd6d-f4056faf38ef None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:50:03.094815 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:50:03.094815 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:50:03.094815 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:50:03.094815 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:50:03.094815 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:50:03.094815 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:50:03.094815 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:50:03.094815 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:50:03.094815 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:50:03.094815 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:50:03.094815 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:50:03.094815 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 19:50:03.094815 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 19:50:03.094815 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 19:50:03.095731 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 19:50:03.095731 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:50:03.095731 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:50:03.095731 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:50:03.095731 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:50:03.095731 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:50:03.095731 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:50:03.095731 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:50:03.095731 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application with Lock( Aug 05 19:50:03.095731 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:50:03.095731 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:50:03.095731 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:50:03.095731 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:50:03.095731 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:50:03.095731 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:50:03.095731 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:50:03.095731 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:50:03.096657 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 19:50:03.096657 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 19:50:03.096657 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:50:03.096657 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Aug 05 19:50:03.096657 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 370/752] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Aug 5 19:50:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:50:03.097456 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d288a9-6987-4695-bd6d-f4056faf38ef None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:03.126144 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:03.148079 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da55cec9-91f3-400d-b49f-6acf8d51bf1f None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:03.182835 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99374350-a8da-4ef0-b581-dba83e9fb643 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:03.183451 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99374350-a8da-4ef0-b581-dba83e9fb643 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:03.183730 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99374350-a8da-4ef0-b581-dba83e9fb643 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:03.184011 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99374350-a8da-4ef0-b581-dba83e9fb643 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:03.207720 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d288a9-6987-4695-bd6d-f4056faf38ef None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:03.236827 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:03.244599 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d47244-5510-4a02-acf8-d44437dc04b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:03.245234 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d47244-5510-4a02-acf8-d44437dc04b4 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:03.245512 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d47244-5510-4a02-acf8-d44437dc04b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:03.245788 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d47244-5510-4a02-acf8-d44437dc04b4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:03.253694 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99374350-a8da-4ef0-b581-dba83e9fb643 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:03.255143 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99374350-a8da-4ef0-b581-dba83e9fb643 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:03.256835 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d47244-5510-4a02-acf8-d44437dc04b4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:03.258428 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d47244-5510-4a02-acf8-d44437dc04b4 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:03.273859 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application [None req-86d47244-5510-4a02-acf8-d44437dc04b4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:50:03.273859 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:50:03.273859 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:50:03.273859 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:50:03.273859 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:50:03.273859 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:50:03.273859 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:50:03.273859 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:50:03.273859 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:50:03.273859 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:50:03.273859 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:50:03.273859 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:50:03.273859 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 19:50:03.273859 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 19:50:03.273859 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 19:50:03.275199 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 19:50:03.275199 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:50:03.275199 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:50:03.275199 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:50:03.275199 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:50:03.275199 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:50:03.275199 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:50:03.275199 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:50:03.275199 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application with Lock( Aug 05 19:50:03.275199 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:50:03.275199 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:50:03.275199 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:50:03.275199 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:50:03.275199 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:50:03.275199 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:50:03.275199 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:50:03.275199 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:50:03.276232 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 19:50:03.276232 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 19:50:03.276232 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:50:03.276232 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Aug 05 19:50:03.276232 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 371/753] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Aug 5 19:50:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:50:03.277745 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dac0ec5-6026-4d27-b820-65b67c3f7e44 None None] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:03.280536 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dac0ec5-6026-4d27-b820-65b67c3f7e44 None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:03.280536 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dac0ec5-6026-4d27-b820-65b67c3f7e44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:03.298227 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.auth.core [None req-4dac0ec5-6026-4d27-b820-65b67c3f7e44 None None] MFA Rules not processed for user `03d1834a64cd4a43829827cb99a72747`. Rule list: `[]` (Enabled: `True`). {{(pid=80364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:50:03.299402 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-4dac0ec5-6026-4d27-b820-65b67c3f7e44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:50:03.374806 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 383/754] 10.222.0.27 () {58 vars in 1282 bytes} [Fri Aug 5 19:50:03 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 05 19:50:03.375675 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99374350-a8da-4ef0-b581-dba83e9fb643 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:03.388772 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 372/755] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Aug 5 19:50:03 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 05 19:50:03.388772 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dac0ec5-6026-4d27-b820-65b67c3f7e44 None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:03.393878 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:03.472941 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:03.493158 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99374350-a8da-4ef0-b581-dba83e9fb643 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:03.499979 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dac0ec5-6026-4d27-b820-65b67c3f7e44 None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:03.529219 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc0424f-8916-4d4d-8ef9-7d60b247ae5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:03.529760 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc0424f-8916-4d4d-8ef9-7d60b247ae5a None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:03.530039 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc0424f-8916-4d4d-8ef9-7d60b247ae5a None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:03.530313 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc0424f-8916-4d4d-8ef9-7d60b247ae5a None admin] PATH_INFO: `/v3/domains` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:03.531000 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc0424f-8916-4d4d-8ef9-7d60b247ae5a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:50:03.531302 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc0424f-8916-4d4d-8ef9-7d60b247ae5a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:03.532678 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc0424f-8916-4d4d-8ef9-7d60b247ae5a None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:03.538113 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7618f42-74a1-4e0e-949d-33cc307c17e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:03.538726 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7618f42-74a1-4e0e-949d-33cc307c17e5 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:03.539312 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7618f42-74a1-4e0e-949d-33cc307c17e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:03.539312 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7618f42-74a1-4e0e-949d-33cc307c17e5 None admin] PATH_INFO: `/v3/domains` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:03.539978 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7618f42-74a1-4e0e-949d-33cc307c17e5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:50:03.540282 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7618f42-74a1-4e0e-949d-33cc307c17e5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:03.541669 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7618f42-74a1-4e0e-949d-33cc307c17e5 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:03.545103 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 384/756] 10.222.0.27 () {56 vars in 1109 bytes} [Fri Aug 5 19:50:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:50:03.545862 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc0424f-8916-4d4d-8ef9-7d60b247ae5a None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:03.561337 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 373/757] 10.222.0.27 () {56 vars in 1109 bytes} [Fri Aug 5 19:50:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:50:03.562534 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7618f42-74a1-4e0e-949d-33cc307c17e5 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:03.628552 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:03.661457 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc0424f-8916-4d4d-8ef9-7d60b247ae5a None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:03.671108 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7618f42-74a1-4e0e-949d-33cc307c17e5 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:03.694621 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12424919-7789-472a-8d72-985157dfed31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:03.695262 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12424919-7789-472a-8d72-985157dfed31 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:03.695545 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12424919-7789-472a-8d72-985157dfed31 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:03.695820 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12424919-7789-472a-8d72-985157dfed31 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:03.706768 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35a940c-0076-4b07-8314-4deeeee3bf0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:03.706943 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12424919-7789-472a-8d72-985157dfed31 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:03.707763 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35a940c-0076-4b07-8314-4deeeee3bf0e None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:03.708238 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35a940c-0076-4b07-8314-4deeeee3bf0e None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:03.708345 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12424919-7789-472a-8d72-985157dfed31 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:03.708809 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35a940c-0076-4b07-8314-4deeeee3bf0e None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:03.709633 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e35a940c-0076-4b07-8314-4deeeee3bf0e None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:03.710810 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:03.712220 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e35a940c-0076-4b07-8314-4deeeee3bf0e None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:03.722408 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application [None req-e35a940c-0076-4b07-8314-4deeeee3bf0e None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 05 19:50:03.722408 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:50:03.722408 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 05 19:50:03.722408 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 05 19:50:03.722408 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 05 19:50:03.722408 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 05 19:50:03.722408 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 05 19:50:03.722408 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 05 19:50:03.722408 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 05 19:50:03.722408 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Aug 05 19:50:03.722408 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 05 19:50:03.722408 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Aug 05 19:50:03.722408 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:50:03.722408 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:50:03.722408 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:50:03.722408 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:50:03.723344 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:50:03.723344 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:50:03.723344 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:50:03.723344 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:50:03.723344 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:50:03.723344 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:50:03.723344 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:50:03.723344 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 05 19:50:03.723344 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 05 19:50:03.723344 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 05 19:50:03.723344 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 05 19:50:03.723344 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:50:03.723344 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:50:03.723344 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:50:03.723344 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:50:03.723344 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:50:03.724190 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:50:03.724190 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:50:03.724190 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application with Lock( Aug 05 19:50:03.724190 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:50:03.724190 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:50:03.724190 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:50:03.724190 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:50:03.724190 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:50:03.724190 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:50:03.724190 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:50:03.724190 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:50:03.724190 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 05 19:50:03.724190 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 05 19:50:03.724190 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 05 19:50:03.724190 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Aug 05 19:50:03.725051 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-12424919-7789-472a-8d72-985157dfed31 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:50:03.725051 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:50:03.725051 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:50:03.725051 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:50:03.725051 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:50:03.725051 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:50:03.725051 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:50:03.725051 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:50:03.725051 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:50:03.725051 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:50:03.725051 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:50:03.725051 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:50:03.725051 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 19:50:03.725051 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 19:50:03.725051 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 19:50:03.725903 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 19:50:03.725903 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:50:03.725903 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:50:03.725903 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:50:03.725903 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:50:03.725903 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:50:03.725903 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:50:03.725903 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:50:03.725903 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Aug 05 19:50:03.725903 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:50:03.725903 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:50:03.725903 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:50:03.725903 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:50:03.725903 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:50:03.725903 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:50:03.725903 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:50:03.725903 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:50:03.726806 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 19:50:03.726806 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 19:50:03.726806 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:50:03.726806 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Aug 05 19:50:03.728745 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 374/758] 10.222.0.27 () {56 vars in 1129 bytes} [Fri Aug 5 19:50:03 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:50:03.729770 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 385/759] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Aug 5 19:50:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:50:03.729986 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35a940c-0076-4b07-8314-4deeeee3bf0e None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:03.730621 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12424919-7789-472a-8d72-985157dfed31 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:03.837240 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12424919-7789-472a-8d72-985157dfed31 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:03.837769 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35a940c-0076-4b07-8314-4deeeee3bf0e None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:03.865571 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:03.875363 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a1baac-7981-4b72-a5a1-415d1ef047eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:03.875563 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01eeabc6-d0d3-48b4-b0cc-974570130a86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:03.876001 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a1baac-7981-4b72-a5a1-415d1ef047eb None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:03.876117 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01eeabc6-d0d3-48b4-b0cc-974570130a86 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:03.876281 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a1baac-7981-4b72-a5a1-415d1ef047eb None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:03.876377 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01eeabc6-d0d3-48b4-b0cc-974570130a86 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:03.876558 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a1baac-7981-4b72-a5a1-415d1ef047eb None admin] PATH_INFO: `/v3/projects/service` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:03.876654 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01eeabc6-d0d3-48b4-b0cc-974570130a86 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:03.883429 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a1baac-7981-4b72-a5a1-415d1ef047eb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:03.884056 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01eeabc6-d0d3-48b4-b0cc-974570130a86 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:03.884890 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a1baac-7981-4b72-a5a1-415d1ef047eb None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:03.885508 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01eeabc6-d0d3-48b4-b0cc-974570130a86 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:03.896314 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application [None req-00a1baac-7981-4b72-a5a1-415d1ef047eb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 19:50:03.896314 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:50:03.896314 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:50:03.896314 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:50:03.896314 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:50:03.896314 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:50:03.896314 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:50:03.896314 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:50:03.896314 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:50:03.896314 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:50:03.896314 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:50:03.896314 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:50:03.896314 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 05 19:50:03.896314 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 05 19:50:03.896314 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 05 19:50:03.897261 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 05 19:50:03.897261 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:50:03.897261 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:50:03.897261 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:50:03.897261 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:50:03.897261 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:50:03.897261 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:50:03.897261 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:50:03.897261 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application with Lock( Aug 05 19:50:03.897261 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:50:03.897261 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:50:03.897261 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:50:03.897261 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:50:03.897261 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:50:03.897261 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:50:03.897261 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:50:03.897261 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:50:03.898143 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 05 19:50:03.898143 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 05 19:50:03.898143 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 05 19:50:03.898143 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 05 19:50:03.898143 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 05 19:50:03.898143 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 05 19:50:03.898143 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 19:50:03.898143 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Aug 05 19:50:03.898585 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-01eeabc6-d0d3-48b4-b0cc-974570130a86 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 19:50:03.898585 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:50:03.898585 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:50:03.898585 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:50:03.898585 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:50:03.898585 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:50:03.898585 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:50:03.898585 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:50:03.898585 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:50:03.898585 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:50:03.898585 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:50:03.898585 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:50:03.898585 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 05 19:50:03.898585 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 05 19:50:03.898585 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 05 19:50:03.899416 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 05 19:50:03.899416 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:50:03.899416 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:50:03.899416 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:50:03.899416 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:50:03.899416 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:50:03.899416 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:50:03.899416 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:50:03.899416 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Aug 05 19:50:03.899416 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:50:03.899416 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:50:03.899416 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:50:03.899416 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:50:03.899416 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:50:03.899416 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:50:03.899416 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:50:03.899416 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:50:03.900623 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 05 19:50:03.900623 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 05 19:50:03.900623 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 05 19:50:03.900623 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 05 19:50:03.900623 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 05 19:50:03.900623 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 05 19:50:03.900623 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 19:50:03.900623 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Aug 05 19:50:03.901056 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 375/760] 10.222.0.27 () {56 vars in 1111 bytes} [Fri Aug 5 19:50:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:50:03.901056 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a1baac-7981-4b72-a5a1-415d1ef047eb None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:03.903000 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 386/761] 10.222.0.27 () {56 vars in 1111 bytes} [Fri Aug 5 19:50:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:50:03.904322 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01eeabc6-d0d3-48b4-b0cc-974570130a86 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:03.965858 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:04.008853 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a1baac-7981-4b72-a5a1-415d1ef047eb None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:04.015888 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01eeabc6-d0d3-48b4-b0cc-974570130a86 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:04.043710 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167c9016-027a-4369-9736-cea746d88931 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:04.044404 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167c9016-027a-4369-9736-cea746d88931 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:04.045230 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167c9016-027a-4369-9736-cea746d88931 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:04.045500 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167c9016-027a-4369-9736-cea746d88931 None admin] PATH_INFO: `/v3/domains` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:04.046297 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-167c9016-027a-4369-9736-cea746d88931 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:50:04.046711 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-167c9016-027a-4369-9736-cea746d88931 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:04.048118 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-167c9016-027a-4369-9736-cea746d88931 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:04.051315 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c8eeec-af97-4273-83e8-d2f5e803b444 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:04.051925 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c8eeec-af97-4273-83e8-d2f5e803b444 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:04.052200 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c8eeec-af97-4273-83e8-d2f5e803b444 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:04.052470 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c8eeec-af97-4273-83e8-d2f5e803b444 None admin] PATH_INFO: `/v3/services` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:04.053169 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c8eeec-af97-4273-83e8-d2f5e803b444 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:50:04.053472 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c8eeec-af97-4273-83e8-d2f5e803b444 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:04.054806 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c8eeec-af97-4273-83e8-d2f5e803b444 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:04.067954 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 387/762] 10.222.0.27 () {56 vars in 1124 bytes} [Fri Aug 5 19:50:03 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:50:04.067954 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c8eeec-af97-4273-83e8-d2f5e803b444 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:04.068255 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 376/763] 10.222.0.27 () {56 vars in 1109 bytes} [Fri Aug 5 19:50:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:50:04.068255 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167c9016-027a-4369-9736-cea746d88931 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:04.116745 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:04.172357 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167c9016-027a-4369-9736-cea746d88931 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:04.172914 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c8eeec-af97-4273-83e8-d2f5e803b444 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:04.193007 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:04.208062 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e55645-c566-4674-b996-39568c36510c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:04.208244 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2f74b4-affa-48b9-bc69-1bba0cecad7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:04.208721 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e55645-c566-4674-b996-39568c36510c None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:04.208893 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2f74b4-affa-48b9-bc69-1bba0cecad7a None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:04.209005 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e55645-c566-4674-b996-39568c36510c None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:04.209155 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2f74b4-affa-48b9-bc69-1bba0cecad7a None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:04.209320 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e55645-c566-4674-b996-39568c36510c None admin] PATH_INFO: `/v3/projects` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:04.209415 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2f74b4-affa-48b9-bc69-1bba0cecad7a None admin] PATH_INFO: `/v3/projects` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:04.210107 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e55645-c566-4674-b996-39568c36510c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:50:04.210213 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce2f74b4-affa-48b9-bc69-1bba0cecad7a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:50:04.210430 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce2f74b4-affa-48b9-bc69-1bba0cecad7a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:04.210532 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e55645-c566-4674-b996-39568c36510c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:04.211860 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce2f74b4-affa-48b9-bc69-1bba0cecad7a None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:04.211963 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e55645-c566-4674-b996-39568c36510c None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:04.229351 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 377/764] 10.222.0.27 () {56 vars in 1148 bytes} [Fri Aug 5 19:50:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:50:04.229532 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 388/765] 10.222.0.27 () {56 vars in 1148 bytes} [Fri Aug 5 19:50:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:50:04.229532 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2f74b4-affa-48b9-bc69-1bba0cecad7a None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:04.230225 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e55645-c566-4674-b996-39568c36510c None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:04.330330 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2f74b4-affa-48b9-bc69-1bba0cecad7a None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:04.330754 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e55645-c566-4674-b996-39568c36510c None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:04.339755 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:04.365991 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c52048-c65d-4c46-8ec7-20dc83791f16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:04.366603 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c52048-c65d-4c46-8ec7-20dc83791f16 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:04.366956 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c52048-c65d-4c46-8ec7-20dc83791f16 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:04.367166 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c52048-c65d-4c46-8ec7-20dc83791f16 None admin] PATH_INFO: `/v3/services` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:04.367847 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c52048-c65d-4c46-8ec7-20dc83791f16 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:50:04.368150 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c52048-c65d-4c46-8ec7-20dc83791f16 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:04.369542 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c52048-c65d-4c46-8ec7-20dc83791f16 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:04.373632 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672f4089-7d4c-4fcc-be14-bc0c5deb82b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:04.374271 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672f4089-7d4c-4fcc-be14-bc0c5deb82b4 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:04.374553 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672f4089-7d4c-4fcc-be14-bc0c5deb82b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:04.374829 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672f4089-7d4c-4fcc-be14-bc0c5deb82b4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:04.374914 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 378/766] 10.222.0.27 () {56 vars in 1124 bytes} [Fri Aug 5 19:50:04 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:50:04.375661 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c52048-c65d-4c46-8ec7-20dc83791f16 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:04.381923 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672f4089-7d4c-4fcc-be14-bc0c5deb82b4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:04.383362 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672f4089-7d4c-4fcc-be14-bc0c5deb82b4 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:04.397420 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-672f4089-7d4c-4fcc-be14-bc0c5deb82b4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 19:50:04.397420 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:50:04.397420 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:50:04.397420 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:50:04.397420 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:50:04.397420 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:50:04.397420 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:50:04.397420 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:50:04.397420 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:50:04.397420 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:50:04.397420 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:50:04.397420 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:50:04.397420 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 05 19:50:04.397420 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 05 19:50:04.397420 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 05 19:50:04.398438 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 05 19:50:04.398438 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:50:04.398438 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:50:04.398438 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:50:04.398438 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:50:04.398438 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:50:04.398438 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:50:04.398438 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:50:04.398438 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Aug 05 19:50:04.398438 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:50:04.398438 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:50:04.398438 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:50:04.398438 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:50:04.398438 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:50:04.398438 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:50:04.398438 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:50:04.398438 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:50:04.404777 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 05 19:50:04.404777 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 05 19:50:04.404777 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 05 19:50:04.404777 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 05 19:50:04.404777 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 05 19:50:04.404777 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 05 19:50:04.404777 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 19:50:04.404777 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Aug 05 19:50:04.404777 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 389/767] 10.222.0.27 () {56 vars in 1111 bytes} [Fri Aug 5 19:50:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:50:04.404777 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672f4089-7d4c-4fcc-be14-bc0c5deb82b4 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:04.441435 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:04.475197 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c52048-c65d-4c46-8ec7-20dc83791f16 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:04.498677 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672f4089-7d4c-4fcc-be14-bc0c5deb82b4 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:04.508451 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbf1c8f-9f4d-4ffb-aaef-57983600877a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:04.509137 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbf1c8f-9f4d-4ffb-aaef-57983600877a None admin] REQUEST_METHOD: `PUT` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:04.509417 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbf1c8f-9f4d-4ffb-aaef-57983600877a None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:04.509685 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbf1c8f-9f4d-4ffb-aaef-57983600877a None admin] PATH_INFO: `/v3/projects/a70a82dcfc574fad90b3f5f00abbba58/users/11001a2ad96f4ec8a35aab2c64376875/roles/c173688a960b46e5be65f0ccd6fbd734` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:04.511622 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffbf1c8f-9f4d-4ffb-aaef-57983600877a None admin] RBAC: Authorizing `identity:create_grant(project_id=a70a82dcfc574fad90b3f5f00abbba58, user_id=11001a2ad96f4ec8a35aab2c64376875, role_id=c173688a960b46e5be65f0ccd6fbd734)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:04.513181 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffbf1c8f-9f4d-4ffb-aaef-57983600877a None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:04.525739 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 379/768] 10.222.0.27 () {58 vars in 1439 bytes} [Fri Aug 5 19:50:04 2022] PUT /identity/v3/projects/a70a82dcfc574fad90b3f5f00abbba58/users/11001a2ad96f4ec8a35aab2c64376875/roles/c173688a960b46e5be65f0ccd6fbd734 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 19:50:04.526916 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f95ea7a-d064-4a14-940d-01f6bbe35d18 None None] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:04.527200 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f95ea7a-d064-4a14-940d-01f6bbe35d18 None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:04.527681 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f95ea7a-d064-4a14-940d-01f6bbe35d18 None None] PATH_INFO: `/` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:04.529307 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 380/769] 10.222.0.27 () {54 vars in 927 bytes} [Fri Aug 5 19:50:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:50:04.530022 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f95ea7a-d064-4a14-940d-01f6bbe35d18 None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:04.533329 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f711b8a-ab79-43ed-ae09-9bdfe0a7136f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:04.533965 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f711b8a-ab79-43ed-ae09-9bdfe0a7136f None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:04.534240 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f711b8a-ab79-43ed-ae09-9bdfe0a7136f None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:04.534508 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f711b8a-ab79-43ed-ae09-9bdfe0a7136f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:04.535319 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f711b8a-ab79-43ed-ae09-9bdfe0a7136f None admin] RBAC: Adding query filter params (role.id=c173688a960b46e5be65f0ccd6fbd734, scope.project.id=a70a82dcfc574fad90b3f5f00abbba58, user.id=6533931da2d34447881008b43372b0d5) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:50:04.535625 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f711b8a-ab79-43ed-ae09-9bdfe0a7136f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:04.537133 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f711b8a-ab79-43ed-ae09-9bdfe0a7136f None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:04.543577 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 390/770] 10.222.0.27 () {56 vars in 1374 bytes} [Fri Aug 5 19:50:04 2022] GET /identity/v3/role_assignments?user.id=6533931da2d34447881008b43372b0d5&scope.project.id=a70a82dcfc574fad90b3f5f00abbba58&role.id=c173688a960b46e5be65f0ccd6fbd734 => generated 259 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:50:04.544270 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f711b8a-ab79-43ed-ae09-9bdfe0a7136f None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:04.596222 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:04.688346 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f95ea7a-d064-4a14-940d-01f6bbe35d18 None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:04.694833 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f711b8a-ab79-43ed-ae09-9bdfe0a7136f None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:04.699245 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:04.727345 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7012fc-807c-4bab-a893-dd3fa3ac867e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:04.728282 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7012fc-807c-4bab-a893-dd3fa3ac867e None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:04.728934 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7012fc-807c-4bab-a893-dd3fa3ac867e None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:04.729387 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7012fc-807c-4bab-a893-dd3fa3ac867e None admin] PATH_INFO: `/v3/endpoints` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:04.730356 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e7012fc-807c-4bab-a893-dd3fa3ac867e None admin] RBAC: Adding query filter params (interface=public, service_id=36f8593a52094a23a45d2693c90fc4c3, region_id=RegionOne) {{(pid=80364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:50:04.730733 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e7012fc-807c-4bab-a893-dd3fa3ac867e None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:04.732240 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e7012fc-807c-4bab-a893-dd3fa3ac867e None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:04.735759 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15db871-aa67-46ef-a034-ff9b68f0e031 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:04.736379 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15db871-aa67-46ef-a034-ff9b68f0e031 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:04.736680 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15db871-aa67-46ef-a034-ff9b68f0e031 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:04.736950 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15db871-aa67-46ef-a034-ff9b68f0e031 None admin] PATH_INFO: `/v3/projects` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:04.737705 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15db871-aa67-46ef-a034-ff9b68f0e031 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:50:04.738012 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15db871-aa67-46ef-a034-ff9b68f0e031 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:04.739479 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15db871-aa67-46ef-a034-ff9b68f0e031 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:04.740701 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 381/771] 10.222.0.27 () {56 vars in 1251 bytes} [Fri Aug 5 19:50:04 2022] GET /identity/v3/endpoints?service_id=36f8593a52094a23a45d2693c90fc4c3&interface=public®ion_id=RegionOne => generated 194 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:50:04.741885 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4213d92-7d23-487e-93f1-5cf8258a8600 None None] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:04.742057 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4213d92-7d23-487e-93f1-5cf8258a8600 None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:04.743117 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4213d92-7d23-487e-93f1-5cf8258a8600 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:04.753063 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 391/772] 10.222.0.27 () {56 vars in 1148 bytes} [Fri Aug 5 19:50:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:50:04.753911 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15db871-aa67-46ef-a034-ff9b68f0e031 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:04.761955 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.auth.core [None req-d4213d92-7d23-487e-93f1-5cf8258a8600 None None] MFA Rules not processed for user `03d1834a64cd4a43829827cb99a72747`. Rule list: `[]` (Enabled: `True`). {{(pid=80364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:50:04.763319 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-d4213d92-7d23-487e-93f1-5cf8258a8600 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:50:04.843519 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 382/773] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Aug 5 19:50:04 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 05 19:50:04.844427 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4213d92-7d23-487e-93f1-5cf8258a8600 None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:04.882888 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15db871-aa67-46ef-a034-ff9b68f0e031 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:04.883315 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:04.933827 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b138f93c-a0b8-4047-bc37-776fb29c793a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:04.934540 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b138f93c-a0b8-4047-bc37-776fb29c793a None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:04.934885 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b138f93c-a0b8-4047-bc37-776fb29c793a None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:04.935283 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b138f93c-a0b8-4047-bc37-776fb29c793a None admin] PATH_INFO: `/v3/services` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:04.936011 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b138f93c-a0b8-4047-bc37-776fb29c793a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:04.937466 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b138f93c-a0b8-4047-bc37-776fb29c793a None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:04.943505 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 392/774] 10.222.0.27 () {56 vars in 1087 bytes} [Fri Aug 5 19:50:04 2022] GET /identity/v3/services => generated 772 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:50:04.944329 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b138f93c-a0b8-4047-bc37-776fb29c793a None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:04.996699 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:04.998906 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4213d92-7d23-487e-93f1-5cf8258a8600 None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:05.073302 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e380a46-b82f-4b2e-a2d5-bdf09fa4e722 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:05.074425 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e380a46-b82f-4b2e-a2d5-bdf09fa4e722 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:05.074975 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e380a46-b82f-4b2e-a2d5-bdf09fa4e722 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:05.075423 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e380a46-b82f-4b2e-a2d5-bdf09fa4e722 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:05.077005 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e380a46-b82f-4b2e-a2d5-bdf09fa4e722 None admin] RBAC: Adding query filter params (role.id=6e2bc1ba438d4eb69923ef4659c181f3, scope.project.id=a70a82dcfc574fad90b3f5f00abbba58, user.id=bdf7cf05a77e4298b2bf4f58b40f54aa) {{(pid=80364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:50:05.077495 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e380a46-b82f-4b2e-a2d5-bdf09fa4e722 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:05.079366 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e380a46-b82f-4b2e-a2d5-bdf09fa4e722 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:05.096970 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 383/775] 10.222.0.27 () {56 vars in 1374 bytes} [Fri Aug 5 19:50:04 2022] GET /identity/v3/role_assignments?user.id=bdf7cf05a77e4298b2bf4f58b40f54aa&scope.project.id=a70a82dcfc574fad90b3f5f00abbba58&role.id=6e2bc1ba438d4eb69923ef4659c181f3 => generated 259 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:50:05.100048 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b138f93c-a0b8-4047-bc37-776fb29c793a None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:05.140257 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353b4c98-1fae-4073-80ab-a0e8db262908 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:05.140982 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353b4c98-1fae-4073-80ab-a0e8db262908 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:05.141263 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353b4c98-1fae-4073-80ab-a0e8db262908 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:05.141537 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353b4c98-1fae-4073-80ab-a0e8db262908 None admin] PATH_INFO: `/v3/services/network` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:05.142188 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353b4c98-1fae-4073-80ab-a0e8db262908 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:05.143598 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353b4c98-1fae-4073-80ab-a0e8db262908 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:05.156084 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-353b4c98-1fae-4073-80ab-a0e8db262908 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 05 19:50:05.156084 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:50:05.156084 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 05 19:50:05.156084 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 05 19:50:05.156084 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 05 19:50:05.156084 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 05 19:50:05.156084 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 05 19:50:05.156084 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 05 19:50:05.156084 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 05 19:50:05.156084 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Aug 05 19:50:05.156084 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 05 19:50:05.156084 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Aug 05 19:50:05.156084 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:50:05.156084 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:50:05.156084 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:50:05.156084 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:50:05.156084 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:50:05.157157 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:50:05.157157 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:50:05.157157 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:50:05.157157 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:50:05.157157 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:50:05.157157 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:50:05.157157 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 05 19:50:05.157157 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 05 19:50:05.157157 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 05 19:50:05.157157 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 05 19:50:05.157157 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:50:05.157157 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:50:05.157157 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:50:05.157157 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:50:05.157157 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:50:05.157157 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:50:05.157984 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:50:05.157984 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Aug 05 19:50:05.157984 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:50:05.157984 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:50:05.157984 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:50:05.157984 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:50:05.157984 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:50:05.157984 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:50:05.157984 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:50:05.157984 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:50:05.157984 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 05 19:50:05.157984 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 05 19:50:05.157984 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 05 19:50:05.157984 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Aug 05 19:50:05.157984 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 393/776] 10.222.0.27 () {56 vars in 1111 bytes} [Fri Aug 5 19:50:04 2022] GET /identity/v3/services/network => generated 88 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:50:05.163606 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e380a46-b82f-4b2e-a2d5-bdf09fa4e722 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:05.267537 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:05.281308 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e380a46-b82f-4b2e-a2d5-bdf09fa4e722 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:05.320634 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90219731-a4c8-48e8-9241-9dff6caaa852 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:05.321330 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90219731-a4c8-48e8-9241-9dff6caaa852 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:05.321611 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90219731-a4c8-48e8-9241-9dff6caaa852 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:05.321881 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90219731-a4c8-48e8-9241-9dff6caaa852 None admin] PATH_INFO: `/v3/services` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:05.322570 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90219731-a4c8-48e8-9241-9dff6caaa852 None admin] RBAC: Adding query filter params (name=network) {{(pid=80364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:50:05.322870 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90219731-a4c8-48e8-9241-9dff6caaa852 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:05.324252 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90219731-a4c8-48e8-9241-9dff6caaa852 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:05.329715 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 384/777] 10.222.0.27 () {56 vars in 1112 bytes} [Fri Aug 5 19:50:05 2022] GET /identity/v3/services?name=network => generated 124 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 19:50:05.334976 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353b4c98-1fae-4073-80ab-a0e8db262908 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:05.454997 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:05.472406 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353b4c98-1fae-4073-80ab-a0e8db262908 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:05.516751 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c67014-6163-42d5-bbe4-0f102115bc1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:05.518218 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c67014-6163-42d5-bbe4-0f102115bc1b None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:05.518715 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c67014-6163-42d5-bbe4-0f102115bc1b None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:05.519182 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c67014-6163-42d5-bbe4-0f102115bc1b None admin] PATH_INFO: `/v3/services` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:05.520062 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c67014-6163-42d5-bbe4-0f102115bc1b None admin] RBAC: Adding query filter params (type=network) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:50:05.520601 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c67014-6163-42d5-bbe4-0f102115bc1b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:05.522214 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c67014-6163-42d5-bbe4-0f102115bc1b None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:05.528973 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 394/778] 10.222.0.27 () {56 vars in 1112 bytes} [Fri Aug 5 19:50:05 2022] GET /identity/v3/services?type=network => generated 350 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 19:50:05.534733 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90219731-a4c8-48e8-9241-9dff6caaa852 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:05.648748 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90219731-a4c8-48e8-9241-9dff6caaa852 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:05.683545 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5fe0724-74a3-4688-ae8b-2fa411c5fab9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:05.684163 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fe0724-74a3-4688-ae8b-2fa411c5fab9 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:05.684439 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fe0724-74a3-4688-ae8b-2fa411c5fab9 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:05.684791 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fe0724-74a3-4688-ae8b-2fa411c5fab9 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:05.685615 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5fe0724-74a3-4688-ae8b-2fa411c5fab9 None admin] RBAC: Adding query filter params (interface=public, service_id=097de8917db44a7d91522db886238865, region_id=RegionOne) {{(pid=80364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:50:05.685922 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5fe0724-74a3-4688-ae8b-2fa411c5fab9 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:05.687288 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5fe0724-74a3-4688-ae8b-2fa411c5fab9 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:05.693217 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 385/779] 10.222.0.27 () {56 vars in 1251 bytes} [Fri Aug 5 19:50:05 2022] GET /identity/v3/endpoints?service_id=097de8917db44a7d91522db886238865&interface=public®ion_id=RegionOne => generated 194 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:50:05.702139 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c67014-6163-42d5-bbe4-0f102115bc1b None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:05.839705 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c67014-6163-42d5-bbe4-0f102115bc1b None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:05.875863 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90571a7e-eb09-4e19-81ff-dffab616f031 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:05.876487 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90571a7e-eb09-4e19-81ff-dffab616f031 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:05.876791 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90571a7e-eb09-4e19-81ff-dffab616f031 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:05.877064 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90571a7e-eb09-4e19-81ff-dffab616f031 None admin] PATH_INFO: `/v3/services` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:05.877789 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90571a7e-eb09-4e19-81ff-dffab616f031 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:05.879149 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90571a7e-eb09-4e19-81ff-dffab616f031 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:05.884810 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 395/780] 10.222.0.27 () {56 vars in 1087 bytes} [Fri Aug 5 19:50:05 2022] GET /identity/v3/services => generated 772 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:50:06.963227 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88071827-2f02-420a-a115-5d3d1b6379dc None None] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:06.964267 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88071827-2f02-420a-a115-5d3d1b6379dc None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:06.964808 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88071827-2f02-420a-a115-5d3d1b6379dc None None] PATH_INFO: `/` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:06.967172 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 386/781] 10.222.0.27 () {54 vars in 927 bytes} [Fri Aug 5 19:50:06 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:50:06.972757 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15287004-bc36-4d2f-ba17-5093bfd00765 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:06.973621 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15287004-bc36-4d2f-ba17-5093bfd00765 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:06.974095 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15287004-bc36-4d2f-ba17-5093bfd00765 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:06.999852 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-15287004-bc36-4d2f-ba17-5093bfd00765 None None] MFA Rules not processed for user `03d1834a64cd4a43829827cb99a72747`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:50:07.001430 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-15287004-bc36-4d2f-ba17-5093bfd00765 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:50:07.024124 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:07.033773 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7e3033-b51e-4b9b-beb3-148194a47fa4 None None] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:07.034057 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7e3033-b51e-4b9b-beb3-148194a47fa4 None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:07.034332 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7e3033-b51e-4b9b-beb3-148194a47fa4 None None] PATH_INFO: `/` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:07.035664 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 387/782] 10.222.0.27 () {54 vars in 927 bytes} [Fri Aug 5 19:50:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:50:07.041297 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7a1149-4734-4308-9dba-026953ed9b6b None None] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:07.041587 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7a1149-4734-4308-9dba-026953ed9b6b None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:07.041867 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7a1149-4734-4308-9dba-026953ed9b6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:07.063296 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.auth.core [None req-5d7a1149-4734-4308-9dba-026953ed9b6b None None] MFA Rules not processed for user `03d1834a64cd4a43829827cb99a72747`. Rule list: `[]` (Enabled: `True`). {{(pid=80364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:50:07.064664 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-5d7a1149-4734-4308-9dba-026953ed9b6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:50:07.092858 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:07.125854 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 396/783] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Aug 5 19:50:06 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 05 19:50:07.157586 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15287004-bc36-4d2f-ba17-5093bfd00765 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:07.172661 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 388/784] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Aug 5 19:50:07 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 05 19:50:07.199638 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7a1149-4734-4308-9dba-026953ed9b6b None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:07.339882 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7a1149-4734-4308-9dba-026953ed9b6b None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:07.382809 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2cc136-f4d5-4dbe-bd95-b799d35e14b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:07.383466 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2cc136-f4d5-4dbe-bd95-b799d35e14b7 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:07.383743 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2cc136-f4d5-4dbe-bd95-b799d35e14b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:07.384008 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2cc136-f4d5-4dbe-bd95-b799d35e14b7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:07.395465 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15287004-bc36-4d2f-ba17-5093bfd00765 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:07.410082 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2cc136-f4d5-4dbe-bd95-b799d35e14b7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 19:50:07.417981 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application [None req-ab2cc136-f4d5-4dbe-bd95-b799d35e14b7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 19:50:07.417981 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:50:07.417981 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:50:07.417981 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:50:07.417981 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:50:07.417981 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:50:07.417981 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:50:07.417981 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:50:07.417981 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:50:07.417981 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:50:07.417981 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:50:07.417981 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:50:07.417981 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 19:50:07.417981 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 19:50:07.417981 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 19:50:07.417981 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 19:50:07.419008 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 19:50:07.419008 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 19:50:07.419008 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 19:50:07.419008 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 19:50:07.419008 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:50:07.419008 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:50:07.419008 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:50:07.419008 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:50:07.419008 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:50:07.419008 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:50:07.419008 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:50:07.419008 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application with Lock( Aug 05 19:50:07.419008 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:50:07.419008 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:50:07.419008 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:50:07.419008 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:50:07.419933 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:50:07.419933 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:50:07.419933 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:50:07.419933 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:50:07.419933 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 19:50:07.419933 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 19:50:07.419933 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 19:50:07.419933 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 19:50:07.419933 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 19:50:07.419933 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 19:50:07.419933 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 19:50:07.419933 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Aug 05 19:50:07.419933 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 389/785] 10.222.0.27 () {56 vars in 1096 bytes} [Fri Aug 5 19:50:07 2022] GET /identity/v3/roles/admin => generated 83 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 05 19:50:07.427006 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2cc136-f4d5-4dbe-bd95-b799d35e14b7 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:07.443730 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:07.456235 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dceda8c-c721-4177-b4c1-952cf8b4af4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:07.457276 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dceda8c-c721-4177-b4c1-952cf8b4af4e None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:07.458411 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dceda8c-c721-4177-b4c1-952cf8b4af4e None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:07.458740 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dceda8c-c721-4177-b4c1-952cf8b4af4e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:07.474830 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dceda8c-c721-4177-b4c1-952cf8b4af4e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 19:50:07.480475 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-1dceda8c-c721-4177-b4c1-952cf8b4af4e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 19:50:07.480475 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:50:07.480475 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:50:07.480475 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:50:07.480475 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:50:07.480475 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:50:07.480475 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:50:07.480475 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:50:07.480475 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:50:07.480475 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:50:07.480475 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:50:07.480475 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:50:07.480475 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 19:50:07.480475 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 19:50:07.480475 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 19:50:07.480475 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 19:50:07.481453 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 19:50:07.481453 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 19:50:07.481453 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 19:50:07.481453 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 19:50:07.481453 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:50:07.481453 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:50:07.481453 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:50:07.481453 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:50:07.481453 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:50:07.481453 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:50:07.481453 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:50:07.481453 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Aug 05 19:50:07.481453 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:50:07.481453 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:50:07.481453 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:50:07.481453 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:50:07.482944 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:50:07.482944 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:50:07.482944 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:50:07.482944 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:50:07.482944 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 19:50:07.482944 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 19:50:07.482944 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 19:50:07.482944 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 19:50:07.482944 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 19:50:07.482944 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 19:50:07.482944 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 19:50:07.482944 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Aug 05 19:50:07.482944 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 397/786] 10.222.0.27 () {56 vars in 1096 bytes} [Fri Aug 5 19:50:07 2022] GET /identity/v3/roles/admin => generated 83 bytes in 325 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:50:07.484195 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c82070-0ddd-4c2c-b542-0383cf3c2272 None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:07.484195 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c82070-0ddd-4c2c-b542-0383cf3c2272 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:07.484344 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c82070-0ddd-4c2c-b542-0383cf3c2272 None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:07.485802 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 398/787] 10.222.0.27 () {54 vars in 927 bytes} [Fri Aug 5 19:50:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:50:07.487022 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa85aaf-396c-4e28-89f7-3c73cac7d5c8 None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:07.487140 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa85aaf-396c-4e28-89f7-3c73cac7d5c8 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:07.487426 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa85aaf-396c-4e28-89f7-3c73cac7d5c8 None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:07.488976 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 399/788] 10.222.0.27 () {54 vars in 927 bytes} [Fri Aug 5 19:50:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:50:07.489765 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa85aaf-396c-4e28-89f7-3c73cac7d5c8 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:07.502603 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:07.562776 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2cc136-f4d5-4dbe-bd95-b799d35e14b7 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:07.595024 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa85aaf-396c-4e28-89f7-3c73cac7d5c8 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:07.603472 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b195057-3d2d-4a64-9952-2b8b70b9dfbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:07.604475 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b195057-3d2d-4a64-9952-2b8b70b9dfbe None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:07.604989 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b195057-3d2d-4a64-9952-2b8b70b9dfbe None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:07.605610 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b195057-3d2d-4a64-9952-2b8b70b9dfbe None admin] PATH_INFO: `/v3/roles` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:07.606396 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b195057-3d2d-4a64-9952-2b8b70b9dfbe None admin] RBAC: Adding query filter params (name=admin) {{(pid=80364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:50:07.606798 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b195057-3d2d-4a64-9952-2b8b70b9dfbe None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:07.608720 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b195057-3d2d-4a64-9952-2b8b70b9dfbe None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:07.622514 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 390/789] 10.222.0.27 () {56 vars in 1099 bytes} [Fri Aug 5 19:50:07 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:50:07.623904 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb76cf80-19e6-40fb-ae16-23a0bfd5eb5d None None] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:07.624180 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb76cf80-19e6-40fb-ae16-23a0bfd5eb5d None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:07.624486 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb76cf80-19e6-40fb-ae16-23a0bfd5eb5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:07.629339 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeca5f43-c51e-4ab9-9c2f-2e0b31da5f15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:07.629962 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeca5f43-c51e-4ab9-9c2f-2e0b31da5f15 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:07.630239 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeca5f43-c51e-4ab9-9c2f-2e0b31da5f15 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:07.630509 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeca5f43-c51e-4ab9-9c2f-2e0b31da5f15 None admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:07.631227 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeca5f43-c51e-4ab9-9c2f-2e0b31da5f15 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:50:07.631533 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeca5f43-c51e-4ab9-9c2f-2e0b31da5f15 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:07.632913 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeca5f43-c51e-4ab9-9c2f-2e0b31da5f15 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:07.641109 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 400/790] 10.222.0.27 () {56 vars in 1099 bytes} [Fri Aug 5 19:50:07 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:50:07.642363 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261477fe-a5a7-4124-8c9d-4434efb80b33 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:07.642649 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261477fe-a5a7-4124-8c9d-4434efb80b33 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:07.642922 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261477fe-a5a7-4124-8c9d-4434efb80b33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:07.646934 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.auth.core [None req-cb76cf80-19e6-40fb-ae16-23a0bfd5eb5d None None] MFA Rules not processed for user `03d1834a64cd4a43829827cb99a72747`. Rule list: `[]` (Enabled: `True`). {{(pid=80364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:50:07.648104 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-cb76cf80-19e6-40fb-ae16-23a0bfd5eb5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:50:07.662741 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-261477fe-a5a7-4124-8c9d-4434efb80b33 None None] MFA Rules not processed for user `03d1834a64cd4a43829827cb99a72747`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:50:07.664195 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-261477fe-a5a7-4124-8c9d-4434efb80b33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:50:07.718126 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:07.732871 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 391/791] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Aug 5 19:50:07 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 05 19:50:07.734138 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb76cf80-19e6-40fb-ae16-23a0bfd5eb5d None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:07.745618 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 401/792] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Aug 5 19:50:07 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 05 19:50:07.746561 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261477fe-a5a7-4124-8c9d-4434efb80b33 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:07.750070 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:07.842913 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb76cf80-19e6-40fb-ae16-23a0bfd5eb5d None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:07.859949 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261477fe-a5a7-4124-8c9d-4434efb80b33 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:07.880597 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55914e0e-9701-4688-843b-fedb1f1fa72a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:07.881518 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55914e0e-9701-4688-843b-fedb1f1fa72a None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:07.881805 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55914e0e-9701-4688-843b-fedb1f1fa72a None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:07.882083 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55914e0e-9701-4688-843b-fedb1f1fa72a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:07.893708 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55914e0e-9701-4688-843b-fedb1f1fa72a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:07.895183 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16098a11-bf75-4193-888b-57955e4e4548 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:07.895309 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55914e0e-9701-4688-843b-fedb1f1fa72a None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:07.895818 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16098a11-bf75-4193-888b-57955e4e4548 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:07.896101 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16098a11-bf75-4193-888b-57955e4e4548 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:07.896373 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16098a11-bf75-4193-888b-57955e4e4548 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:07.907185 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16098a11-bf75-4193-888b-57955e4e4548 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:07.908684 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16098a11-bf75-4193-888b-57955e4e4548 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:07.910733 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application [None req-55914e0e-9701-4688-843b-fedb1f1fa72a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:50:07.910733 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:50:07.910733 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:50:07.910733 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:50:07.910733 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:50:07.910733 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:50:07.910733 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:50:07.910733 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:50:07.910733 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:50:07.910733 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:50:07.910733 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:50:07.910733 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:50:07.910733 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 19:50:07.910733 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 19:50:07.910733 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 19:50:07.911600 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 19:50:07.911600 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:50:07.911600 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:50:07.911600 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:50:07.911600 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:50:07.911600 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:50:07.911600 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:50:07.911600 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:50:07.911600 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application with Lock( Aug 05 19:50:07.911600 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:50:07.911600 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:50:07.911600 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:50:07.911600 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:50:07.911600 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:50:07.911600 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:50:07.911600 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:50:07.911600 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:50:07.912488 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 19:50:07.912488 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 19:50:07.912488 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:50:07.912488 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Aug 05 19:50:07.912488 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 392/793] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Aug 5 19:50:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:50:07.912810 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55914e0e-9701-4688-843b-fedb1f1fa72a None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:07.930172 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-16098a11-bf75-4193-888b-57955e4e4548 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:50:07.930172 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:50:07.930172 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:50:07.930172 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:50:07.930172 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:50:07.930172 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:50:07.930172 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:50:07.930172 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:50:07.930172 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:50:07.930172 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:50:07.930172 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:50:07.930172 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:50:07.930172 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 19:50:07.930172 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 19:50:07.930172 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 19:50:07.931121 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 19:50:07.931121 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:50:07.931121 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:50:07.931121 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:50:07.931121 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:50:07.931121 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:50:07.931121 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:50:07.931121 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:50:07.931121 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Aug 05 19:50:07.931121 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:50:07.931121 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:50:07.931121 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:50:07.931121 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:50:07.931121 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:50:07.931121 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:50:07.931121 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:50:07.931121 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:50:07.932016 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 19:50:07.932016 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 19:50:07.932016 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:50:07.932016 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Aug 05 19:50:07.932016 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 402/794] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Aug 5 19:50:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:50:07.932318 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16098a11-bf75-4193-888b-57955e4e4548 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:07.973128 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:07.992121 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:08.023012 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55914e0e-9701-4688-843b-fedb1f1fa72a None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:08.046649 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16098a11-bf75-4193-888b-57955e4e4548 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:08.059028 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6df7176-96cc-4a12-a568-c3220be34bc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:08.059747 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6df7176-96cc-4a12-a568-c3220be34bc8 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:08.060153 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6df7176-96cc-4a12-a568-c3220be34bc8 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:08.060491 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6df7176-96cc-4a12-a568-c3220be34bc8 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:08.061492 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6df7176-96cc-4a12-a568-c3220be34bc8 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:08.062595 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6df7176-96cc-4a12-a568-c3220be34bc8 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:08.071981 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application [None req-d6df7176-96cc-4a12-a568-c3220be34bc8 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 05 19:50:08.071981 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:50:08.071981 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 05 19:50:08.071981 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 05 19:50:08.071981 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 05 19:50:08.071981 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 05 19:50:08.071981 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 05 19:50:08.071981 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 05 19:50:08.071981 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 05 19:50:08.071981 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Aug 05 19:50:08.071981 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 05 19:50:08.071981 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Aug 05 19:50:08.071981 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:50:08.071981 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:50:08.071981 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:50:08.071981 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:50:08.073589 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:50:08.073589 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:50:08.073589 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:50:08.073589 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:50:08.073589 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:50:08.073589 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:50:08.073589 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:50:08.073589 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 05 19:50:08.073589 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 05 19:50:08.073589 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 05 19:50:08.073589 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 05 19:50:08.073589 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:50:08.073589 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:50:08.073589 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:50:08.073589 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:50:08.073589 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:50:08.074427 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:50:08.074427 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:50:08.074427 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application with Lock( Aug 05 19:50:08.074427 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:50:08.074427 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:50:08.074427 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:50:08.074427 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:50:08.074427 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:50:08.074427 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:50:08.074427 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:50:08.074427 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:50:08.074427 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 05 19:50:08.074427 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 05 19:50:08.074427 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 05 19:50:08.074427 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Aug 05 19:50:08.074427 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 393/795] 10.222.0.27 () {56 vars in 1129 bytes} [Fri Aug 5 19:50:07 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:50:08.075309 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6df7176-96cc-4a12-a568-c3220be34bc8 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:08.091550 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20fe3b4-5d44-4c7d-a1af-1f0540123693 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:08.092199 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20fe3b4-5d44-4c7d-a1af-1f0540123693 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:08.092516 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20fe3b4-5d44-4c7d-a1af-1f0540123693 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:08.092781 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20fe3b4-5d44-4c7d-a1af-1f0540123693 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:08.103865 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f20fe3b4-5d44-4c7d-a1af-1f0540123693 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 19:50:08.109834 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-f20fe3b4-5d44-4c7d-a1af-1f0540123693 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 05 19:50:08.109834 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:50:08.109834 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:50:08.109834 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:50:08.109834 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:50:08.109834 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:50:08.109834 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:50:08.109834 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:50:08.109834 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:50:08.109834 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:50:08.109834 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:50:08.109834 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:50:08.109834 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 19:50:08.109834 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 19:50:08.109834 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 19:50:08.109834 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 19:50:08.110825 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 19:50:08.110825 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 19:50:08.110825 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 19:50:08.110825 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 19:50:08.110825 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:50:08.110825 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:50:08.110825 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:50:08.110825 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:50:08.110825 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:50:08.110825 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:50:08.110825 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:50:08.110825 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Aug 05 19:50:08.110825 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:50:08.110825 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:50:08.110825 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:50:08.110825 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:50:08.111681 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:50:08.111681 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:50:08.111681 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:50:08.111681 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:50:08.111681 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 19:50:08.111681 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 19:50:08.111681 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 19:50:08.111681 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 19:50:08.111681 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 19:50:08.111681 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 19:50:08.111681 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 05 19:50:08.111681 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Aug 05 19:50:08.111681 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 403/796] 10.222.0.27 () {56 vars in 1102 bytes} [Fri Aug 5 19:50:07 2022] GET /identity/v3/roles/service => generated 85 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:50:08.112379 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20fe3b4-5d44-4c7d-a1af-1f0540123693 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:08.191466 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6df7176-96cc-4a12-a568-c3220be34bc8 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:08.207735 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:08.225098 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20fe3b4-5d44-4c7d-a1af-1f0540123693 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:08.226861 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48f537a-be42-49a0-9d50-ea3f96e167da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:08.227496 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48f537a-be42-49a0-9d50-ea3f96e167da None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:08.227777 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48f537a-be42-49a0-9d50-ea3f96e167da None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:08.228056 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48f537a-be42-49a0-9d50-ea3f96e167da None admin] PATH_INFO: `/v3/domains` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:08.228782 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48f537a-be42-49a0-9d50-ea3f96e167da None admin] RBAC: Adding query filter params (name=Default) {{(pid=80364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:50:08.229092 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48f537a-be42-49a0-9d50-ea3f96e167da None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:08.230465 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48f537a-be42-49a0-9d50-ea3f96e167da None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:08.239279 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:08.244128 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 394/797] 10.222.0.27 () {56 vars in 1109 bytes} [Fri Aug 5 19:50:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:50:08.245403 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48f537a-be42-49a0-9d50-ea3f96e167da None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:08.269142 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb5b23a-fec7-4e98-a84d-5632d8705357 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:08.269779 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb5b23a-fec7-4e98-a84d-5632d8705357 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:08.270125 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb5b23a-fec7-4e98-a84d-5632d8705357 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:08.270404 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb5b23a-fec7-4e98-a84d-5632d8705357 None admin] PATH_INFO: `/v3/domains` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:08.271110 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb5b23a-fec7-4e98-a84d-5632d8705357 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:50:08.271903 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb5b23a-fec7-4e98-a84d-5632d8705357 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:08.273630 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb5b23a-fec7-4e98-a84d-5632d8705357 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:08.290321 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 404/798] 10.222.0.27 () {56 vars in 1109 bytes} [Fri Aug 5 19:50:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:50:08.291239 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb5b23a-fec7-4e98-a84d-5632d8705357 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:08.354650 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48f537a-be42-49a0-9d50-ea3f96e167da None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:08.394964 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be5d5aa-b571-4f5f-91ca-9b2634047c8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:08.395646 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be5d5aa-b571-4f5f-91ca-9b2634047c8b None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:08.395932 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be5d5aa-b571-4f5f-91ca-9b2634047c8b None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:08.396204 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be5d5aa-b571-4f5f-91ca-9b2634047c8b None admin] PATH_INFO: `/v3/services` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:08.396924 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be5d5aa-b571-4f5f-91ca-9b2634047c8b None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:50:08.397228 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be5d5aa-b571-4f5f-91ca-9b2634047c8b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:08.398604 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be5d5aa-b571-4f5f-91ca-9b2634047c8b None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:08.400890 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb5b23a-fec7-4e98-a84d-5632d8705357 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:08.404285 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 395/799] 10.222.0.27 () {56 vars in 1124 bytes} [Fri Aug 5 19:50:08 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:50:08.404809 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be5d5aa-b571-4f5f-91ca-9b2634047c8b None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:08.445778 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5120b912-06f4-4410-ac43-9fb237e0b770 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:08.446406 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5120b912-06f4-4410-ac43-9fb237e0b770 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:08.446685 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5120b912-06f4-4410-ac43-9fb237e0b770 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:08.446958 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5120b912-06f4-4410-ac43-9fb237e0b770 None admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:08.447679 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5120b912-06f4-4410-ac43-9fb237e0b770 None admin] RBAC: Adding query filter params (name=service) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:50:08.447985 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5120b912-06f4-4410-ac43-9fb237e0b770 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:08.449791 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5120b912-06f4-4410-ac43-9fb237e0b770 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:08.451349 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:08.460619 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 405/800] 10.222.0.27 () {56 vars in 1103 bytes} [Fri Aug 5 19:50:08 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:50:08.461568 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5120b912-06f4-4410-ac43-9fb237e0b770 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:08.502875 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:08.515496 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be5d5aa-b571-4f5f-91ca-9b2634047c8b None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:08.555743 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba0b8be-80b1-48ee-8ae5-46bddef4da4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:08.556383 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba0b8be-80b1-48ee-8ae5-46bddef4da4a None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:08.556695 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba0b8be-80b1-48ee-8ae5-46bddef4da4a None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:08.556970 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba0b8be-80b1-48ee-8ae5-46bddef4da4a None admin] PATH_INFO: `/v3/users/nova` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:08.564912 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba0b8be-80b1-48ee-8ae5-46bddef4da4a None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:08.566361 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba0b8be-80b1-48ee-8ae5-46bddef4da4a None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:08.578154 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application [None req-fba0b8be-80b1-48ee-8ae5-46bddef4da4a None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 05 19:50:08.578154 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:50:08.578154 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:50:08.578154 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:50:08.578154 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:50:08.578154 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:50:08.578154 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:50:08.578154 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:50:08.578154 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:50:08.578154 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:50:08.578154 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:50:08.578154 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:50:08.578154 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 05 19:50:08.578154 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 05 19:50:08.578154 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 05 19:50:08.578154 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 05 19:50:08.579092 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:50:08.579092 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:50:08.579092 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 05 19:50:08.579092 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 19:50:08.579092 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 05 19:50:08.579092 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 19:50:08.579092 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:50:08.579092 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:50:08.579092 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:50:08.579092 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:50:08.579092 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:50:08.579092 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application with Lock( Aug 05 19:50:08.579092 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:50:08.579092 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:50:08.579092 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:50:08.579092 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:50:08.579092 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:50:08.579092 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:50:08.580046 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:50:08.580046 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:50:08.580046 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 05 19:50:08.580046 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 05 19:50:08.580046 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 05 19:50:08.580046 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 05 19:50:08.580046 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 05 19:50:08.580046 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 05 19:50:08.580046 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 05 19:50:08.580046 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Aug 05 19:50:08.580046 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 396/801] 10.222.0.27 () {56 vars in 1093 bytes} [Fri Aug 5 19:50:08 2022] GET /identity/v3/users/nova => generated 82 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:50:08.580046 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba0b8be-80b1-48ee-8ae5-46bddef4da4a None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:08.581556 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5120b912-06f4-4410-ac43-9fb237e0b770 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:08.619382 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b3588a-eea5-4bc3-8e4d-87c300113ddf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:08.620042 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b3588a-eea5-4bc3-8e4d-87c300113ddf None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:08.620386 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b3588a-eea5-4bc3-8e4d-87c300113ddf None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:08.620787 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b3588a-eea5-4bc3-8e4d-87c300113ddf None admin] PATH_INFO: `/v3/users/swift` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:08.629062 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b3588a-eea5-4bc3-8e4d-87c300113ddf None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:08.630487 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b3588a-eea5-4bc3-8e4d-87c300113ddf None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:08.641445 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-50b3588a-eea5-4bc3-8e4d-87c300113ddf None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 05 19:50:08.641445 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:50:08.641445 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:50:08.641445 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:50:08.641445 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:50:08.641445 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:50:08.641445 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:50:08.641445 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:50:08.641445 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:50:08.641445 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:50:08.641445 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:50:08.641445 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:50:08.641445 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 05 19:50:08.641445 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 05 19:50:08.641445 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 05 19:50:08.641445 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 05 19:50:08.642566 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:50:08.642566 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:50:08.642566 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 05 19:50:08.642566 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 19:50:08.642566 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 05 19:50:08.642566 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 19:50:08.642566 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:50:08.642566 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:50:08.642566 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:50:08.642566 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:50:08.642566 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:50:08.642566 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Aug 05 19:50:08.642566 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:50:08.642566 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:50:08.642566 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:50:08.642566 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:50:08.642566 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:50:08.642566 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:50:08.643534 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:50:08.643534 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:50:08.643534 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 05 19:50:08.643534 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 05 19:50:08.643534 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 05 19:50:08.643534 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 05 19:50:08.643534 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 05 19:50:08.643534 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 05 19:50:08.643534 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 05 19:50:08.643534 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Aug 05 19:50:08.643534 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 406/802] 10.222.0.27 () {56 vars in 1096 bytes} [Fri Aug 5 19:50:08 2022] GET /identity/v3/users/swift => generated 83 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:50:08.643534 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b3588a-eea5-4bc3-8e4d-87c300113ddf None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:08.686898 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:08.690946 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba0b8be-80b1-48ee-8ae5-46bddef4da4a None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:08.723500 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b3c2a2-7f63-4104-ae0c-cddb1dcfe216 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:08.724106 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b3c2a2-7f63-4104-ae0c-cddb1dcfe216 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:08.724382 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b3c2a2-7f63-4104-ae0c-cddb1dcfe216 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:08.724687 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b3c2a2-7f63-4104-ae0c-cddb1dcfe216 None admin] PATH_INFO: `/v3/services` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:08.725352 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b3c2a2-7f63-4104-ae0c-cddb1dcfe216 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:50:08.725650 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b3c2a2-7f63-4104-ae0c-cddb1dcfe216 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:08.727089 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b3c2a2-7f63-4104-ae0c-cddb1dcfe216 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:08.732860 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 397/803] 10.222.0.27 () {56 vars in 1124 bytes} [Fri Aug 5 19:50:08 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:50:08.733644 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b3c2a2-7f63-4104-ae0c-cddb1dcfe216 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:08.744536 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:08.751609 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b3588a-eea5-4bc3-8e4d-87c300113ddf None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:08.784955 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0affe1-eb5d-4c57-bdd4-d4b769fa8a53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:08.785586 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0affe1-eb5d-4c57-bdd4-d4b769fa8a53 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:08.785859 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0affe1-eb5d-4c57-bdd4-d4b769fa8a53 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:08.786129 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0affe1-eb5d-4c57-bdd4-d4b769fa8a53 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:08.797117 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0affe1-eb5d-4c57-bdd4-d4b769fa8a53 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:08.798546 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0affe1-eb5d-4c57-bdd4-d4b769fa8a53 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:08.813425 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-3a0affe1-eb5d-4c57-bdd4-d4b769fa8a53 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:50:08.813425 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:50:08.813425 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:50:08.813425 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:50:08.813425 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:50:08.813425 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:50:08.813425 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:50:08.813425 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:50:08.813425 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:50:08.813425 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:50:08.813425 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:50:08.813425 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:50:08.813425 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 19:50:08.813425 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 19:50:08.813425 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 19:50:08.814457 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 19:50:08.814457 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:50:08.814457 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:50:08.814457 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:50:08.814457 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:50:08.814457 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:50:08.814457 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:50:08.814457 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:50:08.814457 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Aug 05 19:50:08.814457 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:50:08.814457 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:50:08.814457 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:50:08.814457 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:50:08.814457 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:50:08.814457 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:50:08.814457 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:50:08.814457 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:50:08.815387 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 19:50:08.815387 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 19:50:08.815387 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:50:08.815387 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Aug 05 19:50:08.815387 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 407/804] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Aug 5 19:50:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:50:08.815727 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c9a507-ac45-4c7b-bd47-9ab23209d59d None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:08.816021 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c9a507-ac45-4c7b-bd47-9ab23209d59d None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:08.816286 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c9a507-ac45-4c7b-bd47-9ab23209d59d None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:08.817543 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 408/805] 10.222.0.27 () {54 vars in 927 bytes} [Fri Aug 5 19:50:08 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:50:08.818235 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c9a507-ac45-4c7b-bd47-9ab23209d59d None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:08.839841 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b3c2a2-7f63-4104-ae0c-cddb1dcfe216 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:08.873132 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717b7f8f-9099-4db6-b15b-6bc714e8ecd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:08.873681 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717b7f8f-9099-4db6-b15b-6bc714e8ecd4 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:08.873964 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717b7f8f-9099-4db6-b15b-6bc714e8ecd4 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:08.874245 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717b7f8f-9099-4db6-b15b-6bc714e8ecd4 None admin] PATH_INFO: `/v3/users` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:08.875032 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-717b7f8f-9099-4db6-b15b-6bc714e8ecd4 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:50:08.875338 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-717b7f8f-9099-4db6-b15b-6bc714e8ecd4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:08.876797 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-717b7f8f-9099-4db6-b15b-6bc714e8ecd4 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:08.894529 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 398/806] 10.222.0.27 () {56 vars in 1133 bytes} [Fri Aug 5 19:50:08 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:50:08.895302 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717b7f8f-9099-4db6-b15b-6bc714e8ecd4 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:08.918516 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:08.924731 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c9a507-ac45-4c7b-bd47-9ab23209d59d None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:08.957715 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22ddf91-00af-42eb-bb02-5308cd83de8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:08.958354 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22ddf91-00af-42eb-bb02-5308cd83de8e None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:08.958641 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22ddf91-00af-42eb-bb02-5308cd83de8e None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:08.958917 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22ddf91-00af-42eb-bb02-5308cd83de8e None admin] PATH_INFO: `/v3/users` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:08.959712 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d22ddf91-00af-42eb-bb02-5308cd83de8e None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:50:08.960022 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d22ddf91-00af-42eb-bb02-5308cd83de8e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:08.961490 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d22ddf91-00af-42eb-bb02-5308cd83de8e None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:08.978623 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 409/807] 10.222.0.27 () {56 vars in 1135 bytes} [Fri Aug 5 19:50:08 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:50:08.979426 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22ddf91-00af-42eb-bb02-5308cd83de8e None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:08.989423 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:08.996230 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717b7f8f-9099-4db6-b15b-6bc714e8ecd4 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:09.030821 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb8b971-0fb2-4179-8ef3-fed7f5fd02c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:09.031561 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb8b971-0fb2-4179-8ef3-fed7f5fd02c6 None admin] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:09.031851 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb8b971-0fb2-4179-8ef3-fed7f5fd02c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:09.032126 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb8b971-0fb2-4179-8ef3-fed7f5fd02c6 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:09.032884 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb8b971-0fb2-4179-8ef3-fed7f5fd02c6 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:09.034249 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb8b971-0fb2-4179-8ef3-fed7f5fd02c6 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:09.055511 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 399/808] 10.222.0.27 () {60 vars in 1144 bytes} [Fri Aug 5 19:50:08 2022] POST /identity/v3/endpoints => generated 346 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 19:50:09.056654 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31274cd3-05fc-436a-a3e2-dff1f2492e15 None None] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:09.056947 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31274cd3-05fc-436a-a3e2-dff1f2492e15 None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:09.057242 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31274cd3-05fc-436a-a3e2-dff1f2492e15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:09.077476 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.auth.core [None req-31274cd3-05fc-436a-a3e2-dff1f2492e15 None None] MFA Rules not processed for user `03d1834a64cd4a43829827cb99a72747`. Rule list: `[]` (Enabled: `True`). {{(pid=80364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:50:09.078717 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-31274cd3-05fc-436a-a3e2-dff1f2492e15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:50:09.088107 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22ddf91-00af-42eb-bb02-5308cd83de8e None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:09.135516 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a251d167-73e6-4dac-aaae-0fc0f8c01998 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:09.136059 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a251d167-73e6-4dac-aaae-0fc0f8c01998 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:09.136337 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a251d167-73e6-4dac-aaae-0fc0f8c01998 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:09.136642 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a251d167-73e6-4dac-aaae-0fc0f8c01998 None admin] PATH_INFO: `/v3/domains` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:09.137358 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a251d167-73e6-4dac-aaae-0fc0f8c01998 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:50:09.137663 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a251d167-73e6-4dac-aaae-0fc0f8c01998 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:09.139128 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a251d167-73e6-4dac-aaae-0fc0f8c01998 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:09.151680 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 410/809] 10.222.0.27 () {56 vars in 1109 bytes} [Fri Aug 5 19:50:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:50:09.152569 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a251d167-73e6-4dac-aaae-0fc0f8c01998 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:09.158983 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 400/810] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Aug 5 19:50:09 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 05 19:50:09.160121 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31274cd3-05fc-436a-a3e2-dff1f2492e15 None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:09.175947 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:09.240411 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:09.279620 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31274cd3-05fc-436a-a3e2-dff1f2492e15 None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:09.286242 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a251d167-73e6-4dac-aaae-0fc0f8c01998 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:09.313824 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e65e0e-2b33-4603-b9fc-70a7faaed4d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:09.314466 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e65e0e-2b33-4603-b9fc-70a7faaed4d6 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:09.314747 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e65e0e-2b33-4603-b9fc-70a7faaed4d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:09.315019 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e65e0e-2b33-4603-b9fc-70a7faaed4d6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:09.325277 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e65e0e-2b33-4603-b9fc-70a7faaed4d6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:09.326996 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055f3605-ef75-43cd-a10f-bfa4d6da1661 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:09.327265 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e65e0e-2b33-4603-b9fc-70a7faaed4d6 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:09.328116 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055f3605-ef75-43cd-a10f-bfa4d6da1661 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:09.328580 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055f3605-ef75-43cd-a10f-bfa4d6da1661 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:09.329001 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055f3605-ef75-43cd-a10f-bfa4d6da1661 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:09.342875 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application [None req-34e65e0e-2b33-4603-b9fc-70a7faaed4d6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:50:09.342875 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:50:09.342875 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:50:09.342875 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:50:09.342875 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:50:09.342875 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:50:09.342875 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:50:09.342875 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:50:09.342875 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:50:09.342875 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:50:09.342875 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:50:09.342875 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:50:09.342875 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 19:50:09.342875 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 19:50:09.342875 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 19:50:09.343911 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 19:50:09.343911 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:50:09.343911 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:50:09.343911 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:50:09.343911 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:50:09.343911 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:50:09.343911 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:50:09.343911 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:50:09.343911 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application with Lock( Aug 05 19:50:09.343911 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:50:09.343911 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:50:09.343911 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:50:09.343911 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:50:09.343911 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:50:09.343911 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:50:09.343911 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:50:09.343911 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:50:09.344869 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 19:50:09.344869 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 19:50:09.344869 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:50:09.344869 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Aug 05 19:50:09.344869 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 401/811] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Aug 5 19:50:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:50:09.345153 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e65e0e-2b33-4603-b9fc-70a7faaed4d6 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:09.409013 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:09.415523 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-055f3605-ef75-43cd-a10f-bfa4d6da1661 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:09.417007 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-055f3605-ef75-43cd-a10f-bfa4d6da1661 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:09.453062 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e65e0e-2b33-4603-b9fc-70a7faaed4d6 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:09.480840 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:09.492796 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802d6ba2-7c12-4518-b457-08494eeb8885 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:09.493440 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802d6ba2-7c12-4518-b457-08494eeb8885 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:09.493719 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802d6ba2-7c12-4518-b457-08494eeb8885 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:09.493991 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802d6ba2-7c12-4518-b457-08494eeb8885 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:09.502920 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-802d6ba2-7c12-4518-b457-08494eeb8885 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:09.504613 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-802d6ba2-7c12-4518-b457-08494eeb8885 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:09.516910 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application [None req-802d6ba2-7c12-4518-b457-08494eeb8885 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 05 19:50:09.516910 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:50:09.516910 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:50:09.516910 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:50:09.516910 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:50:09.516910 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:50:09.516910 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:50:09.516910 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:50:09.516910 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:50:09.516910 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:50:09.516910 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:50:09.516910 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:50:09.516910 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 05 19:50:09.516910 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 05 19:50:09.516910 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 05 19:50:09.516910 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 05 19:50:09.517889 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:50:09.517889 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:50:09.517889 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 05 19:50:09.517889 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 19:50:09.517889 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 05 19:50:09.517889 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 19:50:09.517889 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:50:09.517889 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:50:09.517889 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:50:09.517889 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:50:09.517889 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:50:09.517889 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application with Lock( Aug 05 19:50:09.517889 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:50:09.517889 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:50:09.517889 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:50:09.517889 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:50:09.517889 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:50:09.517889 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:50:09.518863 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:50:09.518863 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:50:09.518863 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 05 19:50:09.518863 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 05 19:50:09.518863 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 05 19:50:09.518863 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 05 19:50:09.518863 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 05 19:50:09.518863 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 05 19:50:09.518863 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 05 19:50:09.518863 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Aug 05 19:50:09.518863 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 402/812] 10.222.0.27 () {56 vars in 1117 bytes} [Fri Aug 5 19:50:09 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:50:09.518863 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802d6ba2-7c12-4518-b457-08494eeb8885 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:09.540267 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 411/813] 10.222.0.27 () {58 vars in 1282 bytes} [Fri Aug 5 19:50:09 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 384 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 05 19:50:09.540267 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055f3605-ef75-43cd-a10f-bfa4d6da1661 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:09.628602 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802d6ba2-7c12-4518-b457-08494eeb8885 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:09.650843 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:09.656605 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055f3605-ef75-43cd-a10f-bfa4d6da1661 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:09.668223 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f8813c-4d9b-4f9f-8d26-1c1923a5b4bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:09.668828 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f8813c-4d9b-4f9f-8d26-1c1923a5b4bb None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:09.669105 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f8813c-4d9b-4f9f-8d26-1c1923a5b4bb None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:09.669408 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f8813c-4d9b-4f9f-8d26-1c1923a5b4bb None admin] PATH_INFO: `/v3/services/network` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:09.670034 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f8813c-4d9b-4f9f-8d26-1c1923a5b4bb None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:09.671392 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f8813c-4d9b-4f9f-8d26-1c1923a5b4bb None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:09.680078 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application [None req-a7f8813c-4d9b-4f9f-8d26-1c1923a5b4bb None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 05 19:50:09.680078 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:50:09.680078 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 05 19:50:09.680078 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 05 19:50:09.680078 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 05 19:50:09.680078 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 05 19:50:09.680078 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 05 19:50:09.680078 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 05 19:50:09.680078 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 05 19:50:09.680078 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Aug 05 19:50:09.680078 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 05 19:50:09.680078 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Aug 05 19:50:09.680078 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:50:09.680078 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:50:09.680078 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:50:09.680078 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:50:09.680078 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:50:09.681311 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:50:09.681311 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:50:09.681311 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:50:09.681311 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:50:09.681311 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:50:09.681311 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:50:09.681311 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 05 19:50:09.681311 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 05 19:50:09.681311 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 05 19:50:09.681311 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 05 19:50:09.681311 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:50:09.681311 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:50:09.681311 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:50:09.681311 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:50:09.681311 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:50:09.681311 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:50:09.682344 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:50:09.682344 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application with Lock( Aug 05 19:50:09.682344 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:50:09.682344 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:50:09.682344 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:50:09.682344 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:50:09.682344 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:50:09.682344 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:50:09.682344 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:50:09.682344 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:50:09.682344 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 05 19:50:09.682344 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 05 19:50:09.682344 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 05 19:50:09.682344 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Aug 05 19:50:09.686496 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 403/814] 10.222.0.27 () {56 vars in 1111 bytes} [Fri Aug 5 19:50:09 2022] GET /identity/v3/services/network => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:50:09.687385 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f8813c-4d9b-4f9f-8d26-1c1923a5b4bb None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:09.697014 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9691320d-f84e-4c3f-aeac-9151ace4848c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:09.697845 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9691320d-f84e-4c3f-aeac-9151ace4848c None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:09.698306 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9691320d-f84e-4c3f-aeac-9151ace4848c None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:09.698723 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9691320d-f84e-4c3f-aeac-9151ace4848c None admin] PATH_INFO: `/v3/domains` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:09.699597 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9691320d-f84e-4c3f-aeac-9151ace4848c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:50:09.700067 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9691320d-f84e-4c3f-aeac-9151ace4848c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:09.702177 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9691320d-f84e-4c3f-aeac-9151ace4848c None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:09.714472 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 412/815] 10.222.0.27 () {56 vars in 1109 bytes} [Fri Aug 5 19:50:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:50:09.715476 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9691320d-f84e-4c3f-aeac-9151ace4848c None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:09.743537 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:09.798079 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f8813c-4d9b-4f9f-8d26-1c1923a5b4bb None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:09.822978 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9691320d-f84e-4c3f-aeac-9151ace4848c None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:09.834088 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f229f8-67f8-4e94-833f-17771dd1bdd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:09.834700 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f229f8-67f8-4e94-833f-17771dd1bdd4 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:09.834981 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f229f8-67f8-4e94-833f-17771dd1bdd4 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:09.835254 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f229f8-67f8-4e94-833f-17771dd1bdd4 None admin] PATH_INFO: `/v3/users` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:09.836048 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f229f8-67f8-4e94-833f-17771dd1bdd4 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:50:09.836352 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f229f8-67f8-4e94-833f-17771dd1bdd4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:09.838031 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f229f8-67f8-4e94-833f-17771dd1bdd4 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:09.861748 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 404/816] 10.222.0.27 () {56 vars in 1149 bytes} [Fri Aug 5 19:50:09 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:50:09.861748 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f229f8-67f8-4e94-833f-17771dd1bdd4 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:09.864272 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629cb297-7491-41b4-95be-24b9f5d8cc03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:09.864958 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629cb297-7491-41b4-95be-24b9f5d8cc03 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:09.865241 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629cb297-7491-41b4-95be-24b9f5d8cc03 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:09.865512 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629cb297-7491-41b4-95be-24b9f5d8cc03 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:09.879363 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-629cb297-7491-41b4-95be-24b9f5d8cc03 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:09.880933 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-629cb297-7491-41b4-95be-24b9f5d8cc03 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:09.895778 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-629cb297-7491-41b4-95be-24b9f5d8cc03 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:50:09.895778 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:50:09.895778 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:50:09.895778 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:50:09.895778 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:50:09.895778 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:50:09.895778 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:50:09.895778 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:50:09.895778 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:50:09.895778 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:50:09.895778 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:50:09.895778 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:50:09.895778 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 19:50:09.895778 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 19:50:09.895778 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 19:50:09.896936 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 19:50:09.896936 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:50:09.896936 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:50:09.896936 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:50:09.896936 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:50:09.896936 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:50:09.896936 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:50:09.896936 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:50:09.896936 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Aug 05 19:50:09.896936 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:50:09.896936 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:50:09.896936 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:50:09.896936 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:50:09.896936 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:50:09.896936 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:50:09.896936 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:50:09.896936 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:50:09.898024 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 19:50:09.898024 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 19:50:09.898024 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:50:09.898024 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Aug 05 19:50:09.906434 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 413/817] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Aug 5 19:50:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:50:09.906434 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629cb297-7491-41b4-95be-24b9f5d8cc03 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:09.916968 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:09.977270 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:09.980773 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f229f8-67f8-4e94-833f-17771dd1bdd4 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:10.005939 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629cb297-7491-41b4-95be-24b9f5d8cc03 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:10.019724 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41032e6-681c-4c83-b8f1-64b4db3fe267 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:10.020768 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41032e6-681c-4c83-b8f1-64b4db3fe267 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:10.021050 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41032e6-681c-4c83-b8f1-64b4db3fe267 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:10.021591 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41032e6-681c-4c83-b8f1-64b4db3fe267 None admin] PATH_INFO: `/v3/services` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:10.022272 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41032e6-681c-4c83-b8f1-64b4db3fe267 None admin] RBAC: Adding query filter params (name=network) {{(pid=80364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:50:10.022578 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41032e6-681c-4c83-b8f1-64b4db3fe267 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:10.024477 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41032e6-681c-4c83-b8f1-64b4db3fe267 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:10.030763 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 405/818] 10.222.0.27 () {56 vars in 1112 bytes} [Fri Aug 5 19:50:09 2022] GET /identity/v3/services?name=network => generated 124 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:50:10.031894 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41032e6-681c-4c83-b8f1-64b4db3fe267 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:10.043343 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a433de21-f6d4-4ac8-a2fa-232b917ad50b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:10.044348 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a433de21-f6d4-4ac8-a2fa-232b917ad50b None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:10.044667 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a433de21-f6d4-4ac8-a2fa-232b917ad50b None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:10.044945 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a433de21-f6d4-4ac8-a2fa-232b917ad50b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:10.052162 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a433de21-f6d4-4ac8-a2fa-232b917ad50b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:10.053629 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a433de21-f6d4-4ac8-a2fa-232b917ad50b None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:10.065042 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-a433de21-f6d4-4ac8-a2fa-232b917ad50b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 19:50:10.065042 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:50:10.065042 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:50:10.065042 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:50:10.065042 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:50:10.065042 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:50:10.065042 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:50:10.065042 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:50:10.065042 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:50:10.065042 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:50:10.065042 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:50:10.065042 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:50:10.065042 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 05 19:50:10.065042 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 05 19:50:10.065042 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 05 19:50:10.065985 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 05 19:50:10.065985 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:50:10.065985 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:50:10.065985 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:50:10.065985 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:50:10.065985 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:50:10.065985 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:50:10.065985 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:50:10.065985 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Aug 05 19:50:10.065985 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:50:10.065985 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:50:10.065985 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:50:10.065985 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:50:10.065985 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:50:10.065985 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:50:10.065985 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:50:10.065985 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:50:10.071988 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 05 19:50:10.071988 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 05 19:50:10.071988 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 05 19:50:10.071988 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 05 19:50:10.071988 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 05 19:50:10.071988 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 05 19:50:10.071988 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 19:50:10.071988 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Aug 05 19:50:10.071988 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 414/819] 10.222.0.27 () {56 vars in 1111 bytes} [Fri Aug 5 19:50:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:50:10.071988 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a433de21-f6d4-4ac8-a2fa-232b917ad50b None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:10.150905 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41032e6-681c-4c83-b8f1-64b4db3fe267 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:10.160104 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:10.186346 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a433de21-f6d4-4ac8-a2fa-232b917ad50b None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:10.191834 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fadb46b-a4e8-4843-8a3f-4c3fd45fb3e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:10.192485 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fadb46b-a4e8-4843-8a3f-4c3fd45fb3e2 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:10.192807 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fadb46b-a4e8-4843-8a3f-4c3fd45fb3e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:10.193082 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fadb46b-a4e8-4843-8a3f-4c3fd45fb3e2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:10.204548 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fadb46b-a4e8-4843-8a3f-4c3fd45fb3e2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:10.205988 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fadb46b-a4e8-4843-8a3f-4c3fd45fb3e2 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:10.222492 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application [None req-4fadb46b-a4e8-4843-8a3f-4c3fd45fb3e2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:50:10.222492 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:50:10.222492 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:50:10.222492 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:50:10.222492 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:50:10.222492 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:50:10.222492 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:50:10.222492 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:50:10.222492 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:50:10.222492 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:50:10.222492 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:50:10.222492 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:50:10.222492 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 19:50:10.222492 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 19:50:10.222492 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 19:50:10.223387 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 19:50:10.223387 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:50:10.223387 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:50:10.223387 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:50:10.223387 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:50:10.223387 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:50:10.223387 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:50:10.223387 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:50:10.223387 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application with Lock( Aug 05 19:50:10.223387 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:50:10.223387 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:50:10.223387 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:50:10.223387 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:50:10.223387 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:50:10.223387 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:50:10.223387 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:50:10.223387 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:50:10.224274 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 19:50:10.224274 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 19:50:10.224274 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:50:10.224274 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Aug 05 19:50:10.224274 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 406/820] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Aug 5 19:50:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:50:10.225112 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fadb46b-a4e8-4843-8a3f-4c3fd45fb3e2 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:10.229082 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641a5453-dd27-41ab-affb-197187bd6f9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:10.229655 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641a5453-dd27-41ab-affb-197187bd6f9f None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:10.229932 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641a5453-dd27-41ab-affb-197187bd6f9f None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:10.230200 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641a5453-dd27-41ab-affb-197187bd6f9f None admin] PATH_INFO: `/v3/domains` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:10.230901 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-641a5453-dd27-41ab-affb-197187bd6f9f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:50:10.231204 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-641a5453-dd27-41ab-affb-197187bd6f9f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:10.232620 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-641a5453-dd27-41ab-affb-197187bd6f9f None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:10.234400 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:10.251107 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 415/821] 10.222.0.27 () {56 vars in 1109 bytes} [Fri Aug 5 19:50:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:50:10.251107 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641a5453-dd27-41ab-affb-197187bd6f9f None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:10.333303 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fadb46b-a4e8-4843-8a3f-4c3fd45fb3e2 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:10.355855 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641a5453-dd27-41ab-affb-197187bd6f9f None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:10.372535 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c905ce8c-1e0b-4918-a18a-0136c3343b38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:10.373217 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c905ce8c-1e0b-4918-a18a-0136c3343b38 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:10.373432 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c905ce8c-1e0b-4918-a18a-0136c3343b38 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:10.373730 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c905ce8c-1e0b-4918-a18a-0136c3343b38 None admin] PATH_INFO: `/v3/services` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:10.374415 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c905ce8c-1e0b-4918-a18a-0136c3343b38 None admin] RBAC: Adding query filter params (type=network) {{(pid=80364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:50:10.374708 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c905ce8c-1e0b-4918-a18a-0136c3343b38 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:10.376070 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c905ce8c-1e0b-4918-a18a-0136c3343b38 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:10.381422 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 407/822] 10.222.0.27 () {56 vars in 1112 bytes} [Fri Aug 5 19:50:10 2022] GET /identity/v3/services?type=network => generated 350 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:50:10.382003 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c905ce8c-1e0b-4918-a18a-0136c3343b38 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:10.401563 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d79ee0-8729-4021-98d5-55d0b1341667 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:10.401967 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d79ee0-8729-4021-98d5-55d0b1341667 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:10.402249 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d79ee0-8729-4021-98d5-55d0b1341667 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:10.402525 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d79ee0-8729-4021-98d5-55d0b1341667 None admin] PATH_INFO: `/v3/projects` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:10.403273 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d79ee0-8729-4021-98d5-55d0b1341667 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:50:10.403668 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d79ee0-8729-4021-98d5-55d0b1341667 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:10.405138 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d79ee0-8729-4021-98d5-55d0b1341667 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:10.407821 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:10.417069 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 416/823] 10.222.0.27 () {56 vars in 1148 bytes} [Fri Aug 5 19:50:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:50:10.417710 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d79ee0-8729-4021-98d5-55d0b1341667 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:10.476574 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:10.496070 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c905ce8c-1e0b-4918-a18a-0136c3343b38 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:10.526630 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d79ee0-8729-4021-98d5-55d0b1341667 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:10.531529 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d77d430-beb6-48e4-81b2-228cf0a29bb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:10.532244 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d77d430-beb6-48e4-81b2-228cf0a29bb0 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:10.532662 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d77d430-beb6-48e4-81b2-228cf0a29bb0 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:10.533032 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d77d430-beb6-48e4-81b2-228cf0a29bb0 None admin] PATH_INFO: `/v3/domains` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:10.533821 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d77d430-beb6-48e4-81b2-228cf0a29bb0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:50:10.534219 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d77d430-beb6-48e4-81b2-228cf0a29bb0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:10.535645 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d77d430-beb6-48e4-81b2-228cf0a29bb0 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:10.552002 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 408/824] 10.222.0.27 () {56 vars in 1109 bytes} [Fri Aug 5 19:50:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:50:10.552002 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d77d430-beb6-48e4-81b2-228cf0a29bb0 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:10.568805 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010c1cfb-1acb-412b-9eaa-c1941cdba9b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:10.569531 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010c1cfb-1acb-412b-9eaa-c1941cdba9b4 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:10.569870 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010c1cfb-1acb-412b-9eaa-c1941cdba9b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:10.570501 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010c1cfb-1acb-412b-9eaa-c1941cdba9b4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:10.577493 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-010c1cfb-1acb-412b-9eaa-c1941cdba9b4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:10.579021 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-010c1cfb-1acb-412b-9eaa-c1941cdba9b4 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:10.590584 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-010c1cfb-1acb-412b-9eaa-c1941cdba9b4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 19:50:10.590584 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:50:10.590584 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:50:10.590584 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:50:10.590584 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:50:10.590584 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:50:10.590584 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:50:10.590584 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:50:10.590584 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:50:10.590584 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:50:10.590584 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:50:10.590584 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:50:10.590584 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 05 19:50:10.590584 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 05 19:50:10.590584 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 05 19:50:10.592180 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 05 19:50:10.592180 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:50:10.592180 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:50:10.592180 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:50:10.592180 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:50:10.592180 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:50:10.592180 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:50:10.592180 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:50:10.592180 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Aug 05 19:50:10.592180 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:50:10.592180 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:50:10.592180 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:50:10.592180 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:50:10.592180 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:50:10.592180 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:50:10.592180 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:50:10.592180 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:50:10.593315 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 05 19:50:10.593315 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 05 19:50:10.593315 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 05 19:50:10.593315 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 05 19:50:10.593315 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 05 19:50:10.593315 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 05 19:50:10.593315 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 19:50:10.593315 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Aug 05 19:50:10.598708 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 417/825] 10.222.0.27 () {56 vars in 1111 bytes} [Fri Aug 5 19:50:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:50:10.598708 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010c1cfb-1acb-412b-9eaa-c1941cdba9b4 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:10.642456 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:10.663414 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d77d430-beb6-48e4-81b2-228cf0a29bb0 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:10.703143 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010c1cfb-1acb-412b-9eaa-c1941cdba9b4 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:10.704683 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a69bc84-7701-4106-ad0d-b34548cd4ab9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:10.706049 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a69bc84-7701-4106-ad0d-b34548cd4ab9 None admin] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:10.706049 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a69bc84-7701-4106-ad0d-b34548cd4ab9 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:10.706049 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a69bc84-7701-4106-ad0d-b34548cd4ab9 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:10.706814 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a69bc84-7701-4106-ad0d-b34548cd4ab9 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:10.708233 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a69bc84-7701-4106-ad0d-b34548cd4ab9 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:10.719249 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:10.740532 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 409/826] 10.222.0.27 () {60 vars in 1144 bytes} [Fri Aug 5 19:50:10 2022] POST /identity/v3/endpoints => generated 337 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 19:50:10.740532 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a69bc84-7701-4106-ad0d-b34548cd4ab9 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:10.763984 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e1e008-8ed9-423c-955a-30eeba3da9b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:10.764618 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e1e008-8ed9-423c-955a-30eeba3da9b4 None admin] REQUEST_METHOD: `PUT` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:10.764898 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e1e008-8ed9-423c-955a-30eeba3da9b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:10.765170 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e1e008-8ed9-423c-955a-30eeba3da9b4 None admin] PATH_INFO: `/v3/projects/a70a82dcfc574fad90b3f5f00abbba58/users/6533931da2d34447881008b43372b0d5/roles/c173688a960b46e5be65f0ccd6fbd734` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:10.767355 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e1e008-8ed9-423c-955a-30eeba3da9b4 None admin] RBAC: Authorizing `identity:create_grant(project_id=a70a82dcfc574fad90b3f5f00abbba58, user_id=6533931da2d34447881008b43372b0d5, role_id=c173688a960b46e5be65f0ccd6fbd734)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:10.768935 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e1e008-8ed9-423c-955a-30eeba3da9b4 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:10.784835 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 418/827] 10.222.0.27 () {58 vars in 1439 bytes} [Fri Aug 5 19:50:10 2022] PUT /identity/v3/projects/a70a82dcfc574fad90b3f5f00abbba58/users/6533931da2d34447881008b43372b0d5/roles/c173688a960b46e5be65f0ccd6fbd734 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 19:50:10.785747 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e1e008-8ed9-423c-955a-30eeba3da9b4 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:10.876209 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a69bc84-7701-4106-ad0d-b34548cd4ab9 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:10.957377 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f6c03b-81a1-47b9-99c6-39abc3fe37f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:10.958049 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f6c03b-81a1-47b9-99c6-39abc3fe37f5 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:10.958325 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f6c03b-81a1-47b9-99c6-39abc3fe37f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:10.958595 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f6c03b-81a1-47b9-99c6-39abc3fe37f5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:10.976962 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f6c03b-81a1-47b9-99c6-39abc3fe37f5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:10.978457 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f6c03b-81a1-47b9-99c6-39abc3fe37f5 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:10.980996 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e1e008-8ed9-423c-955a-30eeba3da9b4 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:10.982382 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:11.005127 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application [None req-83f6c03b-81a1-47b9-99c6-39abc3fe37f5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 19:50:11.005127 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:50:11.005127 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:50:11.005127 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:50:11.005127 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:50:11.005127 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:50:11.005127 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:50:11.005127 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:50:11.005127 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:50:11.005127 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:50:11.005127 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:50:11.005127 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:50:11.005127 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 05 19:50:11.005127 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 05 19:50:11.005127 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 05 19:50:11.006261 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 05 19:50:11.006261 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:50:11.006261 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:50:11.006261 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:50:11.006261 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:50:11.006261 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:50:11.006261 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:50:11.006261 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:50:11.006261 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application with Lock( Aug 05 19:50:11.006261 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:50:11.006261 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:50:11.006261 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:50:11.006261 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:50:11.006261 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:50:11.006261 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:50:11.006261 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:50:11.006261 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:50:11.007589 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 05 19:50:11.007589 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 05 19:50:11.007589 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 05 19:50:11.007589 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 05 19:50:11.007589 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 05 19:50:11.007589 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 05 19:50:11.007589 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 19:50:11.007589 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Aug 05 19:50:11.007589 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 410/828] 10.222.0.27 () {56 vars in 1111 bytes} [Fri Aug 5 19:50:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 275 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:50:11.011827 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f6c03b-81a1-47b9-99c6-39abc3fe37f5 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:11.042977 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e7552d-0596-42a8-b9c0-72fc5f5c2a1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:11.043595 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e7552d-0596-42a8-b9c0-72fc5f5c2a1a None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:11.043874 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e7552d-0596-42a8-b9c0-72fc5f5c2a1a None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:11.044717 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e7552d-0596-42a8-b9c0-72fc5f5c2a1a None admin] PATH_INFO: `/v3/projects` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:11.046028 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e7552d-0596-42a8-b9c0-72fc5f5c2a1a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:50:11.046336 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e7552d-0596-42a8-b9c0-72fc5f5c2a1a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:11.048254 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e7552d-0596-42a8-b9c0-72fc5f5c2a1a None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:11.068064 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 419/829] 10.222.0.27 () {56 vars in 1148 bytes} [Fri Aug 5 19:50:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:50:11.077160 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e7552d-0596-42a8-b9c0-72fc5f5c2a1a None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:11.120344 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:11.144781 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f6c03b-81a1-47b9-99c6-39abc3fe37f5 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:11.187826 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e379a738-b9f4-4127-bbf4-6ac2d9386fab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:11.188448 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e379a738-b9f4-4127-bbf4-6ac2d9386fab None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:11.188772 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e379a738-b9f4-4127-bbf4-6ac2d9386fab None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:11.189046 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e379a738-b9f4-4127-bbf4-6ac2d9386fab None admin] PATH_INFO: `/v3/projects` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:11.189798 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e379a738-b9f4-4127-bbf4-6ac2d9386fab None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:50:11.190107 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e379a738-b9f4-4127-bbf4-6ac2d9386fab None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:11.191539 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e379a738-b9f4-4127-bbf4-6ac2d9386fab None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:11.200201 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e7552d-0596-42a8-b9c0-72fc5f5c2a1a None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:11.207571 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 411/830] 10.222.0.27 () {56 vars in 1148 bytes} [Fri Aug 5 19:50:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 19:50:11.207571 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d25810e-f6cc-4570-a87b-76a26f3e0c87 None None] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:11.207571 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d25810e-f6cc-4570-a87b-76a26f3e0c87 None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:11.207571 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d25810e-f6cc-4570-a87b-76a26f3e0c87 None None] PATH_INFO: `/` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:11.207571 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 412/831] 10.222.0.27 () {54 vars in 927 bytes} [Fri Aug 5 19:50:11 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:50:11.212045 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d25810e-f6cc-4570-a87b-76a26f3e0c87 None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:11.243567 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029edda1-396e-41ea-b004-eb8a2ca5ff10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:11.244181 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029edda1-396e-41ea-b004-eb8a2ca5ff10 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:11.244460 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029edda1-396e-41ea-b004-eb8a2ca5ff10 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:11.244763 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029edda1-396e-41ea-b004-eb8a2ca5ff10 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:11.245562 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-029edda1-396e-41ea-b004-eb8a2ca5ff10 None admin] RBAC: Adding query filter params (role.id=c173688a960b46e5be65f0ccd6fbd734, scope.project.id=a70a82dcfc574fad90b3f5f00abbba58, user.id=11001a2ad96f4ec8a35aab2c64376875) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:50:11.245883 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-029edda1-396e-41ea-b004-eb8a2ca5ff10 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:11.248230 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-029edda1-396e-41ea-b004-eb8a2ca5ff10 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:11.254451 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 420/832] 10.222.0.27 () {56 vars in 1374 bytes} [Fri Aug 5 19:50:11 2022] GET /identity/v3/role_assignments?user.id=11001a2ad96f4ec8a35aab2c64376875&scope.project.id=a70a82dcfc574fad90b3f5f00abbba58&role.id=c173688a960b46e5be65f0ccd6fbd734 => generated 608 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:50:11.255721 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2d4b57-94f8-4c1d-9efb-b84c2a75ce3c None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:11.256004 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2d4b57-94f8-4c1d-9efb-b84c2a75ce3c None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:11.256280 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2d4b57-94f8-4c1d-9efb-b84c2a75ce3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:11.278365 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-ab2d4b57-94f8-4c1d-9efb-b84c2a75ce3c None None] MFA Rules not processed for user `03d1834a64cd4a43829827cb99a72747`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:50:11.280538 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-ab2d4b57-94f8-4c1d-9efb-b84c2a75ce3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:50:11.286960 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:11.350304 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d25810e-f6cc-4570-a87b-76a26f3e0c87 None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:11.399111 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 421/833] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Aug 5 19:50:11 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 05 19:50:11.402326 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d39d9e-fdbb-447a-b79b-a7482f8bd02a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:11.403022 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d39d9e-fdbb-447a-b79b-a7482f8bd02a None admin] REQUEST_METHOD: `PUT` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:11.403306 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d39d9e-fdbb-447a-b79b-a7482f8bd02a None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:11.403619 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d39d9e-fdbb-447a-b79b-a7482f8bd02a None admin] PATH_INFO: `/v3/projects/a70a82dcfc574fad90b3f5f00abbba58/users/bdf7cf05a77e4298b2bf4f58b40f54aa/roles/6e2bc1ba438d4eb69923ef4659c181f3` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:11.416336 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2d4b57-94f8-4c1d-9efb-b84c2a75ce3c None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:11.420834 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d39d9e-fdbb-447a-b79b-a7482f8bd02a None admin] RBAC: Authorizing `identity:create_grant(project_id=a70a82dcfc574fad90b3f5f00abbba58, user_id=bdf7cf05a77e4298b2bf4f58b40f54aa, role_id=6e2bc1ba438d4eb69923ef4659c181f3)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:11.422316 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d39d9e-fdbb-447a-b79b-a7482f8bd02a None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:11.438843 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 413/834] 10.222.0.27 () {58 vars in 1439 bytes} [Fri Aug 5 19:50:11 2022] PUT /identity/v3/projects/a70a82dcfc574fad90b3f5f00abbba58/users/bdf7cf05a77e4298b2bf4f58b40f54aa/roles/6e2bc1ba438d4eb69923ef4659c181f3 => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 19:50:11.444977 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:11.546839 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2d4b57-94f8-4c1d-9efb-b84c2a75ce3c None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:11.586041 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbbba576-8f2b-4c19-9608-b3d2bb8bce28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:11.586797 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbba576-8f2b-4c19-9608-b3d2bb8bce28 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:11.587087 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbba576-8f2b-4c19-9608-b3d2bb8bce28 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:11.587400 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbba576-8f2b-4c19-9608-b3d2bb8bce28 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:11.588051 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbbba576-8f2b-4c19-9608-b3d2bb8bce28 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:11.589556 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbbba576-8f2b-4c19-9608-b3d2bb8bce28 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:11.599410 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-fbbba576-8f2b-4c19-9608-b3d2bb8bce28 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 05 19:50:11.599410 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:50:11.599410 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 05 19:50:11.599410 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 05 19:50:11.599410 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 05 19:50:11.599410 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 05 19:50:11.599410 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 05 19:50:11.599410 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 05 19:50:11.599410 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 05 19:50:11.599410 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Aug 05 19:50:11.599410 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 05 19:50:11.599410 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Aug 05 19:50:11.599410 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:50:11.599410 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:50:11.599410 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:50:11.599410 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:50:11.599410 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:50:11.600717 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:50:11.600717 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:50:11.600717 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:50:11.600717 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:50:11.600717 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:50:11.600717 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:50:11.600717 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 05 19:50:11.600717 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 05 19:50:11.600717 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 05 19:50:11.600717 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 05 19:50:11.600717 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:50:11.600717 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:50:11.600717 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:50:11.600717 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:50:11.600717 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:50:11.600717 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:50:11.601708 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:50:11.601708 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Aug 05 19:50:11.601708 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:50:11.601708 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:50:11.601708 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:50:11.601708 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:50:11.601708 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:50:11.601708 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:50:11.601708 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:50:11.601708 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:50:11.601708 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 05 19:50:11.601708 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 05 19:50:11.601708 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 05 19:50:11.601708 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Aug 05 19:50:11.601708 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 422/835] 10.222.0.27 () {56 vars in 1114 bytes} [Fri Aug 5 19:50:11 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 05 19:50:11.607556 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d39d9e-fdbb-447a-b79b-a7482f8bd02a None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:11.719125 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d39d9e-fdbb-447a-b79b-a7482f8bd02a None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:11.729000 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:11.757231 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f619e779-db1e-4016-8f77-102dd36b8610 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:11.757854 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f619e779-db1e-4016-8f77-102dd36b8610 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:11.758135 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f619e779-db1e-4016-8f77-102dd36b8610 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:11.758409 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f619e779-db1e-4016-8f77-102dd36b8610 None admin] PATH_INFO: `/v3/services` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:11.759090 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f619e779-db1e-4016-8f77-102dd36b8610 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:50:11.759393 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f619e779-db1e-4016-8f77-102dd36b8610 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:11.760976 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f619e779-db1e-4016-8f77-102dd36b8610 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:11.766844 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 414/836] 10.222.0.27 () {56 vars in 1114 bytes} [Fri Aug 5 19:50:11 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:50:11.772466 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbbba576-8f2b-4c19-9608-b3d2bb8bce28 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:11.865347 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:11.876957 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbbba576-8f2b-4c19-9608-b3d2bb8bce28 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:11.912086 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ada21d-332a-4efa-bda5-723e2eeeb693 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:11.912788 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ada21d-332a-4efa-bda5-723e2eeeb693 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:11.913074 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ada21d-332a-4efa-bda5-723e2eeeb693 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:11.913343 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ada21d-332a-4efa-bda5-723e2eeeb693 None admin] PATH_INFO: `/v3/services` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:11.914031 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ada21d-332a-4efa-bda5-723e2eeeb693 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:50:11.914332 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ada21d-332a-4efa-bda5-723e2eeeb693 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:11.915700 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ada21d-332a-4efa-bda5-723e2eeeb693 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:11.924901 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 423/837] 10.222.0.27 () {56 vars in 1114 bytes} [Fri Aug 5 19:50:11 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:50:12.943342 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5762a00d-d1d4-4864-bc3f-7dfa392d1d98 None None] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:12.944017 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5762a00d-d1d4-4864-bc3f-7dfa392d1d98 None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:12.944017 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5762a00d-d1d4-4864-bc3f-7dfa392d1d98 None None] PATH_INFO: `/` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:12.945685 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 415/838] 10.222.0.27 () {54 vars in 927 bytes} [Fri Aug 5 19:50:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:50:12.951002 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f574910-9fd8-4943-8fd5-9e7ea7f21bba None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:12.951511 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f574910-9fd8-4943-8fd5-9e7ea7f21bba None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:12.951671 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f574910-9fd8-4943-8fd5-9e7ea7f21bba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:12.970275 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-3f574910-9fd8-4943-8fd5-9e7ea7f21bba None None] MFA Rules not processed for user `03d1834a64cd4a43829827cb99a72747`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:50:12.971450 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-3f574910-9fd8-4943-8fd5-9e7ea7f21bba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:50:13.044689 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 424/839] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Aug 5 19:50:12 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 05 19:50:13.064489 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5762a00d-d1d4-4864-bc3f-7dfa392d1d98 None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:13.176563 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5762a00d-d1d4-4864-bc3f-7dfa392d1d98 None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:13.212803 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aeac04d-e19d-4665-b6c4-7f3b85f60ceb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:13.213368 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aeac04d-e19d-4665-b6c4-7f3b85f60ceb None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:13.213650 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aeac04d-e19d-4665-b6c4-7f3b85f60ceb None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:13.213925 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aeac04d-e19d-4665-b6c4-7f3b85f60ceb None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:13.224639 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aeac04d-e19d-4665-b6c4-7f3b85f60ceb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 19:50:13.229474 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application [None req-9aeac04d-e19d-4665-b6c4-7f3b85f60ceb None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 19:50:13.229474 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:50:13.229474 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:50:13.229474 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:50:13.229474 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:50:13.229474 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:50:13.229474 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:50:13.229474 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:50:13.229474 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:50:13.229474 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:50:13.229474 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:50:13.229474 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:50:13.229474 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 19:50:13.229474 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 19:50:13.229474 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 19:50:13.229474 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 19:50:13.230405 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 19:50:13.230405 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 19:50:13.230405 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 19:50:13.230405 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 19:50:13.230405 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:50:13.230405 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:50:13.230405 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:50:13.230405 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:50:13.230405 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:50:13.230405 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:50:13.230405 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:50:13.230405 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application with Lock( Aug 05 19:50:13.230405 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:50:13.230405 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:50:13.230405 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:50:13.230405 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:50:13.231231 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:50:13.231231 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:50:13.231231 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:50:13.231231 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:50:13.231231 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 19:50:13.231231 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 19:50:13.231231 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 19:50:13.231231 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 19:50:13.231231 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 19:50:13.231231 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 19:50:13.231231 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 19:50:13.231231 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Aug 05 19:50:13.231231 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 416/840] 10.222.0.27 () {56 vars in 1096 bytes} [Fri Aug 5 19:50:13 2022] GET /identity/v3/roles/admin => generated 83 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:50:13.237445 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f574910-9fd8-4943-8fd5-9e7ea7f21bba None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:13.321144 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:13.339431 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f574910-9fd8-4943-8fd5-9e7ea7f21bba None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:13.375099 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90925d1d-9d6f-492f-83ce-110d6a4faca0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:13.375800 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90925d1d-9d6f-492f-83ce-110d6a4faca0 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:13.376151 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90925d1d-9d6f-492f-83ce-110d6a4faca0 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:13.376423 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90925d1d-9d6f-492f-83ce-110d6a4faca0 None admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:13.377183 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90925d1d-9d6f-492f-83ce-110d6a4faca0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:50:13.377487 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90925d1d-9d6f-492f-83ce-110d6a4faca0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:13.378850 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90925d1d-9d6f-492f-83ce-110d6a4faca0 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:13.387638 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 425/841] 10.222.0.27 () {56 vars in 1099 bytes} [Fri Aug 5 19:50:13 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:50:13.397850 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aeac04d-e19d-4665-b6c4-7f3b85f60ceb None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:13.432553 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:13.463647 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a688a442-ee2d-4201-b206-04519a789d60 None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:13.464199 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a688a442-ee2d-4201-b206-04519a789d60 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:13.464693 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a688a442-ee2d-4201-b206-04519a789d60 None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:13.467170 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 426/842] 10.222.0.27 () {54 vars in 927 bytes} [Fri Aug 5 19:50:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 05 19:50:13.471938 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9ac87a-c3b7-4b82-8375-be0255ea9956 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:13.472226 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9ac87a-c3b7-4b82-8375-be0255ea9956 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:13.472524 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9ac87a-c3b7-4b82-8375-be0255ea9956 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:13.493530 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-7a9ac87a-c3b7-4b82-8375-be0255ea9956 None None] MFA Rules not processed for user `03d1834a64cd4a43829827cb99a72747`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:50:13.495166 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-7a9ac87a-c3b7-4b82-8375-be0255ea9956 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:50:13.514766 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aeac04d-e19d-4665-b6c4-7f3b85f60ceb None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:13.552051 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474ab8d4-594c-4a75-8114-8aeef88f0104 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:13.552721 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474ab8d4-594c-4a75-8114-8aeef88f0104 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:13.553003 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474ab8d4-594c-4a75-8114-8aeef88f0104 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:13.553276 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474ab8d4-594c-4a75-8114-8aeef88f0104 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:13.565342 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474ab8d4-594c-4a75-8114-8aeef88f0104 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:13.566902 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474ab8d4-594c-4a75-8114-8aeef88f0104 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:13.571325 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 427/843] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Aug 5 19:50:13 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 05 19:50:13.585232 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application [None req-474ab8d4-594c-4a75-8114-8aeef88f0104 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:50:13.585232 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:50:13.585232 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:50:13.585232 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:50:13.585232 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:50:13.585232 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:50:13.585232 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:50:13.585232 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:50:13.585232 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:50:13.585232 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:50:13.585232 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:50:13.585232 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:50:13.585232 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 19:50:13.585232 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 19:50:13.585232 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 19:50:13.586129 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 19:50:13.586129 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:50:13.586129 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:50:13.586129 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:50:13.586129 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:50:13.586129 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:50:13.586129 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:50:13.586129 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:50:13.586129 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application with Lock( Aug 05 19:50:13.586129 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:50:13.586129 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:50:13.586129 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:50:13.586129 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:50:13.586129 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:50:13.586129 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:50:13.586129 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:50:13.586129 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:50:13.587045 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 19:50:13.587045 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 19:50:13.587045 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:50:13.587045 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Aug 05 19:50:13.593692 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 417/844] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Aug 5 19:50:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:50:13.594679 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9ac87a-c3b7-4b82-8375-be0255ea9956 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:13.595998 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474ab8d4-594c-4a75-8114-8aeef88f0104 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:13.677333 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:13.685787 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:13.703548 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474ab8d4-594c-4a75-8114-8aeef88f0104 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:13.706847 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9ac87a-c3b7-4b82-8375-be0255ea9956 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:13.739834 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d3f083-c6f8-4d25-96e0-5dadcdf475d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:13.740379 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d3f083-c6f8-4d25-96e0-5dadcdf475d8 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:13.740687 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d3f083-c6f8-4d25-96e0-5dadcdf475d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:13.740962 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d3f083-c6f8-4d25-96e0-5dadcdf475d8 None admin] PATH_INFO: `/v3/domains` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:13.741697 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d3f083-c6f8-4d25-96e0-5dadcdf475d8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:50:13.742044 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d3f083-c6f8-4d25-96e0-5dadcdf475d8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:13.742865 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930d060e-813c-4280-b32b-c9669b8773bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:13.743413 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d3f083-c6f8-4d25-96e0-5dadcdf475d8 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:13.743509 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930d060e-813c-4280-b32b-c9669b8773bc None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:13.743771 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930d060e-813c-4280-b32b-c9669b8773bc None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:13.744044 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930d060e-813c-4280-b32b-c9669b8773bc None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:13.744783 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930d060e-813c-4280-b32b-c9669b8773bc None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:13.746187 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930d060e-813c-4280-b32b-c9669b8773bc None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:13.755071 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-930d060e-813c-4280-b32b-c9669b8773bc None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 05 19:50:13.755071 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:50:13.755071 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 05 19:50:13.755071 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 05 19:50:13.755071 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 05 19:50:13.755071 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 05 19:50:13.755071 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 05 19:50:13.755071 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 05 19:50:13.755071 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 05 19:50:13.755071 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Aug 05 19:50:13.755071 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 05 19:50:13.755071 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Aug 05 19:50:13.755071 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:50:13.755071 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:50:13.755071 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:50:13.755071 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:50:13.756026 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:50:13.756026 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:50:13.756026 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:50:13.756026 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:50:13.756026 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:50:13.756026 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:50:13.756026 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:50:13.756026 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 05 19:50:13.756026 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 05 19:50:13.756026 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 05 19:50:13.756026 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 05 19:50:13.756026 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:50:13.756026 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:50:13.756026 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:50:13.756026 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:50:13.756026 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:50:13.762404 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:50:13.762404 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:50:13.762404 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Aug 05 19:50:13.762404 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:50:13.762404 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:50:13.762404 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:50:13.762404 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:50:13.762404 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:50:13.762404 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:50:13.762404 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:50:13.762404 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:50:13.762404 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 05 19:50:13.762404 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 05 19:50:13.762404 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 05 19:50:13.762404 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Aug 05 19:50:13.762404 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 428/846] 10.222.0.27 () {56 vars in 1132 bytes} [Fri Aug 5 19:50:13 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 05 19:50:13.763776 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afcac29-c2fc-4f6e-bb7a-0a4127983462 None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:13.763776 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afcac29-c2fc-4f6e-bb7a-0a4127983462 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:13.763776 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afcac29-c2fc-4f6e-bb7a-0a4127983462 None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:13.764730 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 418/846] 10.222.0.27 () {56 vars in 1109 bytes} [Fri Aug 5 19:50:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 19:50:13.764730 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d3f083-c6f8-4d25-96e0-5dadcdf475d8 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:13.764886 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 429/847] 10.222.0.27 () {54 vars in 927 bytes} [Fri Aug 5 19:50:13 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:50:13.765110 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0afcac29-c2fc-4f6e-bb7a-0a4127983462 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:13.866143 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d3f083-c6f8-4d25-96e0-5dadcdf475d8 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:13.874468 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0afcac29-c2fc-4f6e-bb7a-0a4127983462 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:13.903614 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07229dc9-01f5-41b9-9516-c5e64724b077 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:13.904216 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07229dc9-01f5-41b9-9516-c5e64724b077 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:13.904518 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07229dc9-01f5-41b9-9516-c5e64724b077 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:13.904966 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07229dc9-01f5-41b9-9516-c5e64724b077 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:13.910355 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964fe5bb-7bcb-440f-b8e2-64c1699477ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:13.910972 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964fe5bb-7bcb-440f-b8e2-64c1699477ed None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:13.911253 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964fe5bb-7bcb-440f-b8e2-64c1699477ed None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:13.911522 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964fe5bb-7bcb-440f-b8e2-64c1699477ed None admin] PATH_INFO: `/v3/services` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:13.912201 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-964fe5bb-7bcb-440f-b8e2-64c1699477ed None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:50:13.912452 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07229dc9-01f5-41b9-9516-c5e64724b077 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:13.912588 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-964fe5bb-7bcb-440f-b8e2-64c1699477ed None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:13.913921 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07229dc9-01f5-41b9-9516-c5e64724b077 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:13.914095 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-964fe5bb-7bcb-440f-b8e2-64c1699477ed None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:13.923785 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 430/848] 10.222.0.27 () {56 vars in 1126 bytes} [Fri Aug 5 19:50:13 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:50:13.925090 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea657a2-de80-4391-b9ee-5c28609efba5 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:13.925381 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea657a2-de80-4391-b9ee-5c28609efba5 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:13.925660 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea657a2-de80-4391-b9ee-5c28609efba5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:13.927381 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application [None req-07229dc9-01f5-41b9-9516-c5e64724b077 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 05 19:50:13.927381 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:50:13.927381 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:50:13.927381 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:50:13.927381 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:50:13.927381 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:50:13.927381 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:50:13.927381 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:50:13.927381 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:50:13.927381 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:50:13.927381 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:50:13.927381 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:50:13.927381 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 05 19:50:13.927381 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 05 19:50:13.927381 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 05 19:50:13.927381 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 05 19:50:13.928280 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:50:13.928280 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:50:13.928280 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 05 19:50:13.928280 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 19:50:13.928280 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 05 19:50:13.928280 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 19:50:13.928280 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:50:13.928280 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:50:13.928280 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:50:13.928280 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:50:13.928280 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:50:13.928280 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application with Lock( Aug 05 19:50:13.928280 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:50:13.928280 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:50:13.928280 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:50:13.928280 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:50:13.928280 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:50:13.928280 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:50:13.929267 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:50:13.929267 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:50:13.929267 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 05 19:50:13.929267 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 05 19:50:13.929267 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 05 19:50:13.929267 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 05 19:50:13.929267 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 05 19:50:13.929267 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 05 19:50:13.929267 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 05 19:50:13.929267 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Aug 05 19:50:13.933327 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 419/849] 10.222.0.27 () {56 vars in 1096 bytes} [Fri Aug 5 19:50:13 2022] GET /identity/v3/users/swift => generated 83 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 05 19:50:13.934027 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07229dc9-01f5-41b9-9516-c5e64724b077 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:13.935752 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:13.938504 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:13.950880 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-cea657a2-de80-4391-b9ee-5c28609efba5 None None] MFA Rules not processed for user `03d1834a64cd4a43829827cb99a72747`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:50:13.952086 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-cea657a2-de80-4391-b9ee-5c28609efba5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:50:14.027596 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 431/850] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Aug 5 19:50:13 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 05 19:50:14.028421 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea657a2-de80-4391-b9ee-5c28609efba5 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:14.046055 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07229dc9-01f5-41b9-9516-c5e64724b077 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:14.083317 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1de69f-6bf2-40b8-91ec-bfb8b286727d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:14.083937 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1de69f-6bf2-40b8-91ec-bfb8b286727d None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:14.084217 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1de69f-6bf2-40b8-91ec-bfb8b286727d None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:14.084490 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1de69f-6bf2-40b8-91ec-bfb8b286727d None admin] PATH_INFO: `/v3/services` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:14.085195 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db1de69f-6bf2-40b8-91ec-bfb8b286727d None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:50:14.085501 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db1de69f-6bf2-40b8-91ec-bfb8b286727d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:14.086878 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db1de69f-6bf2-40b8-91ec-bfb8b286727d None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:14.091849 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 420/851] 10.222.0.27 () {56 vars in 1126 bytes} [Fri Aug 5 19:50:13 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:50:14.093165 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfdc29b-1408-4f67-b2a9-d607ad6b25fc None None] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:14.093438 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfdc29b-1408-4f67-b2a9-d607ad6b25fc None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:14.093718 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfdc29b-1408-4f67-b2a9-d607ad6b25fc None None] PATH_INFO: `/` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:14.095131 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 421/852] 10.222.0.27 () {54 vars in 927 bytes} [Fri Aug 5 19:50:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:50:14.095756 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcfdc29b-1408-4f67-b2a9-d607ad6b25fc None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:14.150961 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea657a2-de80-4391-b9ee-5c28609efba5 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:14.183476 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:14.186862 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:14.193697 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa3eb61-db1c-42c9-b139-c5b1ae157545 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:14.194332 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa3eb61-db1c-42c9-b139-c5b1ae157545 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:14.194612 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa3eb61-db1c-42c9-b139-c5b1ae157545 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:14.194992 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa3eb61-db1c-42c9-b139-c5b1ae157545 None admin] PATH_INFO: `/v3/users` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:14.195833 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa3eb61-db1c-42c9-b139-c5b1ae157545 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:50:14.196141 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa3eb61-db1c-42c9-b139-c5b1ae157545 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:14.197669 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa3eb61-db1c-42c9-b139-c5b1ae157545 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:14.220174 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 432/853] 10.222.0.27 () {56 vars in 1135 bytes} [Fri Aug 5 19:50:14 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:50:14.221511 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00a9a0c-e8e5-485e-8965-0dbb483af29e None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:14.221800 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00a9a0c-e8e5-485e-8965-0dbb483af29e None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:14.222074 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00a9a0c-e8e5-485e-8965-0dbb483af29e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:14.235431 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcfdc29b-1408-4f67-b2a9-d607ad6b25fc None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:14.242723 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-e00a9a0c-e8e5-485e-8965-0dbb483af29e None None] MFA Rules not processed for user `03d1834a64cd4a43829827cb99a72747`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:50:14.243937 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-e00a9a0c-e8e5-485e-8965-0dbb483af29e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:50:14.279884 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15afc94c-0391-41eb-9008-f50141bdf74f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:14.280442 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15afc94c-0391-41eb-9008-f50141bdf74f None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:14.280749 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15afc94c-0391-41eb-9008-f50141bdf74f None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:14.281020 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15afc94c-0391-41eb-9008-f50141bdf74f None admin] PATH_INFO: `/v3/roles/service` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:14.293216 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15afc94c-0391-41eb-9008-f50141bdf74f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 19:50:14.299960 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application [None req-15afc94c-0391-41eb-9008-f50141bdf74f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 05 19:50:14.299960 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:50:14.299960 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:50:14.299960 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:50:14.299960 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:50:14.299960 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:50:14.299960 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:50:14.299960 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:50:14.299960 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:50:14.299960 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:50:14.299960 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:50:14.299960 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:50:14.299960 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 19:50:14.299960 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 19:50:14.299960 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 19:50:14.299960 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 19:50:14.301140 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 19:50:14.301140 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 19:50:14.301140 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 19:50:14.301140 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 19:50:14.301140 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:50:14.301140 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:50:14.301140 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:50:14.301140 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:50:14.301140 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:50:14.301140 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:50:14.301140 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:50:14.301140 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application with Lock( Aug 05 19:50:14.301140 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:50:14.301140 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:50:14.301140 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:50:14.301140 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:50:14.302155 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:50:14.302155 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:50:14.302155 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:50:14.302155 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:50:14.302155 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 19:50:14.302155 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 19:50:14.302155 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 19:50:14.302155 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 19:50:14.302155 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 19:50:14.302155 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 19:50:14.302155 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 05 19:50:14.302155 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Aug 05 19:50:14.303478 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 422/854] 10.222.0.27 () {56 vars in 1102 bytes} [Fri Aug 5 19:50:14 2022] GET /identity/v3/roles/service => generated 85 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:50:14.304137 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15afc94c-0391-41eb-9008-f50141bdf74f None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:14.332651 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 433/855] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Aug 5 19:50:14 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 05 19:50:14.334342 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00a9a0c-e8e5-485e-8965-0dbb483af29e None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:14.432695 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15afc94c-0391-41eb-9008-f50141bdf74f None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:14.450599 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00a9a0c-e8e5-485e-8965-0dbb483af29e None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:14.461975 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:14.473234 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac046a0f-5d73-4acc-bb9f-cf21cd4ef194 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:14.473852 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:14.474874 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac046a0f-5d73-4acc-bb9f-cf21cd4ef194 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:14.475519 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac046a0f-5d73-4acc-bb9f-cf21cd4ef194 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:14.475733 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac046a0f-5d73-4acc-bb9f-cf21cd4ef194 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:14.488844 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c432c33-866d-4c04-af93-82fc0ee34987 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:14.489540 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c432c33-866d-4c04-af93-82fc0ee34987 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:14.489884 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c432c33-866d-4c04-af93-82fc0ee34987 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:14.490236 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c432c33-866d-4c04-af93-82fc0ee34987 None admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:14.491022 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c432c33-866d-4c04-af93-82fc0ee34987 None admin] RBAC: Adding query filter params (name=service) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:50:14.491386 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c432c33-866d-4c04-af93-82fc0ee34987 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:14.492826 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c432c33-866d-4c04-af93-82fc0ee34987 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:14.501884 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 434/856] 10.222.0.27 () {56 vars in 1103 bytes} [Fri Aug 5 19:50:14 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:50:14.503203 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c432c33-866d-4c04-af93-82fc0ee34987 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:14.551674 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac046a0f-5d73-4acc-bb9f-cf21cd4ef194 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:14.553202 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac046a0f-5d73-4acc-bb9f-cf21cd4ef194 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:14.619924 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c432c33-866d-4c04-af93-82fc0ee34987 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:14.656232 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 423/857] 10.222.0.27 () {58 vars in 1282 bytes} [Fri Aug 5 19:50:14 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 05 19:50:14.657106 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac046a0f-5d73-4acc-bb9f-cf21cd4ef194 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:14.661300 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1432593-bc0d-49c0-a09d-a647927701e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:14.661999 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1432593-bc0d-49c0-a09d-a647927701e2 None admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:14.662348 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1432593-bc0d-49c0-a09d-a647927701e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:14.662686 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1432593-bc0d-49c0-a09d-a647927701e2 None admin] PATH_INFO: `/v3/services` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:14.663448 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1432593-bc0d-49c0-a09d-a647927701e2 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:14.664908 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1432593-bc0d-49c0-a09d-a647927701e2 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:14.680720 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 435/858] 10.222.0.27 () {60 vars in 1141 bytes} [Fri Aug 5 19:50:14 2022] POST /identity/v3/services => generated 251 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 19:50:14.681639 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1432593-bc0d-49c0-a09d-a647927701e2 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:14.692865 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:14.753299 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:14.784640 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac046a0f-5d73-4acc-bb9f-cf21cd4ef194 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:14.829450 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1432593-bc0d-49c0-a09d-a647927701e2 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:14.831612 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f93da88-ff8b-481e-b9d1-fbbbcd87a6c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:14.832469 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f93da88-ff8b-481e-b9d1-fbbbcd87a6c0 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:14.832912 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f93da88-ff8b-481e-b9d1-fbbbcd87a6c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:14.833312 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f93da88-ff8b-481e-b9d1-fbbbcd87a6c0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:14.845269 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f93da88-ff8b-481e-b9d1-fbbbcd87a6c0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:14.847150 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f93da88-ff8b-481e-b9d1-fbbbcd87a6c0 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:14.864339 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application [None req-5f93da88-ff8b-481e-b9d1-fbbbcd87a6c0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:50:14.864339 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:50:14.864339 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:50:14.864339 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:50:14.864339 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:50:14.864339 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:50:14.864339 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:50:14.864339 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:50:14.864339 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:50:14.864339 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:50:14.864339 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:50:14.864339 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:50:14.864339 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 19:50:14.864339 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 19:50:14.864339 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 19:50:14.865474 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 19:50:14.865474 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:50:14.865474 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:50:14.865474 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:50:14.865474 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:50:14.865474 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:50:14.865474 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:50:14.865474 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:50:14.865474 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application with Lock( Aug 05 19:50:14.865474 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:50:14.865474 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:50:14.865474 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:50:14.865474 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:50:14.865474 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:50:14.865474 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:50:14.865474 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:50:14.865474 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:50:14.866457 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 19:50:14.866457 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 19:50:14.866457 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:50:14.866457 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Aug 05 19:50:14.868116 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 424/859] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Aug 5 19:50:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:50:14.871529 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef7fa75-710c-4baa-a1c7-8f1b23f7f012 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:14.872231 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef7fa75-710c-4baa-a1c7-8f1b23f7f012 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:14.872541 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef7fa75-710c-4baa-a1c7-8f1b23f7f012 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:14.873010 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef7fa75-710c-4baa-a1c7-8f1b23f7f012 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:14.874175 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f93da88-ff8b-481e-b9d1-fbbbcd87a6c0 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:14.886303 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef7fa75-710c-4baa-a1c7-8f1b23f7f012 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:14.887673 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef7fa75-710c-4baa-a1c7-8f1b23f7f012 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:14.904307 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-eef7fa75-710c-4baa-a1c7-8f1b23f7f012 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:50:14.904307 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:50:14.904307 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:50:14.904307 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:50:14.904307 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:50:14.904307 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:50:14.904307 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:50:14.904307 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:50:14.904307 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:50:14.904307 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:50:14.904307 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:50:14.904307 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:50:14.904307 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 19:50:14.904307 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 19:50:14.904307 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 19:50:14.905260 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 19:50:14.905260 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:50:14.905260 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:50:14.905260 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:50:14.905260 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:50:14.905260 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:50:14.905260 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:50:14.905260 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:50:14.905260 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Aug 05 19:50:14.905260 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:50:14.905260 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:50:14.905260 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:50:14.905260 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:50:14.905260 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:50:14.905260 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:50:14.905260 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:50:14.905260 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:50:14.906125 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 19:50:14.906125 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 19:50:14.906125 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:50:14.906125 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Aug 05 19:50:14.906125 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 436/860] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Aug 5 19:50:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:50:14.912165 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef7fa75-710c-4baa-a1c7-8f1b23f7f012 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:14.984853 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:15.010029 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f93da88-ff8b-481e-b9d1-fbbbcd87a6c0 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:15.012546 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:15.025179 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef7fa75-710c-4baa-a1c7-8f1b23f7f012 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:15.059696 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85445756-5b43-467a-9bd3-6e7d5e3bb507 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:15.060997 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85445756-5b43-467a-9bd3-6e7d5e3bb507 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:15.061420 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85445756-5b43-467a-9bd3-6e7d5e3bb507 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:15.061933 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85445756-5b43-467a-9bd3-6e7d5e3bb507 None admin] PATH_INFO: `/v3/domains` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:15.062530 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22505690-54ae-4ab8-8032-ed4ce82cba44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:15.062982 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85445756-5b43-467a-9bd3-6e7d5e3bb507 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:50:15.063324 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22505690-54ae-4ab8-8032-ed4ce82cba44 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:15.063595 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85445756-5b43-467a-9bd3-6e7d5e3bb507 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:15.063789 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22505690-54ae-4ab8-8032-ed4ce82cba44 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:15.064154 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22505690-54ae-4ab8-8032-ed4ce82cba44 None admin] PATH_INFO: `/v3/domains` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:15.064974 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22505690-54ae-4ab8-8032-ed4ce82cba44 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:50:15.065381 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22505690-54ae-4ab8-8032-ed4ce82cba44 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:15.065788 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85445756-5b43-467a-9bd3-6e7d5e3bb507 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:15.066903 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22505690-54ae-4ab8-8032-ed4ce82cba44 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:15.088081 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 437/862] 10.222.0.27 () {56 vars in 1109 bytes} [Fri Aug 5 19:50:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 19:50:15.088081 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22505690-54ae-4ab8-8032-ed4ce82cba44 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:15.088531 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 425/862] 10.222.0.27 () {56 vars in 1109 bytes} [Fri Aug 5 19:50:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:50:15.090502 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85445756-5b43-467a-9bd3-6e7d5e3bb507 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:15.207795 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22505690-54ae-4ab8-8032-ed4ce82cba44 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:15.233149 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85445756-5b43-467a-9bd3-6e7d5e3bb507 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:15.254743 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7803ce43-c803-488e-956d-3f2d5eea09dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:15.255766 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7803ce43-c803-488e-956d-3f2d5eea09dd None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:15.256141 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7803ce43-c803-488e-956d-3f2d5eea09dd None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:15.256730 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7803ce43-c803-488e-956d-3f2d5eea09dd None admin] PATH_INFO: `/v3/projects/service` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:15.266767 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7803ce43-c803-488e-956d-3f2d5eea09dd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:15.269024 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7803ce43-c803-488e-956d-3f2d5eea09dd None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:15.283604 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87cccd87-3d87-44f9-94d1-0a5bb2da237f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:15.284398 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cccd87-3d87-44f9-94d1-0a5bb2da237f None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:15.284802 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cccd87-3d87-44f9-94d1-0a5bb2da237f None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:15.285167 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cccd87-3d87-44f9-94d1-0a5bb2da237f None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:15.285285 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-7803ce43-c803-488e-956d-3f2d5eea09dd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 19:50:15.285285 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:50:15.285285 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:50:15.285285 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:50:15.285285 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:50:15.285285 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:50:15.285285 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:50:15.285285 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:50:15.285285 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:50:15.285285 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:50:15.285285 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:50:15.285285 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:50:15.285285 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 05 19:50:15.285285 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 05 19:50:15.285285 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 05 19:50:15.286294 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 05 19:50:15.286294 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:50:15.286294 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:50:15.286294 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:50:15.286294 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:50:15.286294 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:50:15.286294 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:50:15.286294 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:50:15.286294 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Aug 05 19:50:15.286294 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:50:15.286294 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:50:15.286294 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:50:15.286294 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:50:15.286294 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:50:15.286294 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:50:15.286294 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:50:15.286294 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:50:15.287181 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 05 19:50:15.287181 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 05 19:50:15.287181 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 05 19:50:15.287181 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 05 19:50:15.287181 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 05 19:50:15.287181 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 05 19:50:15.287181 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 19:50:15.287181 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Aug 05 19:50:15.287181 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 438/863] 10.222.0.27 () {56 vars in 1111 bytes} [Fri Aug 5 19:50:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:50:15.291958 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7803ce43-c803-488e-956d-3f2d5eea09dd None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:15.293825 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:15.301285 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87cccd87-3d87-44f9-94d1-0a5bb2da237f None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:15.302958 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87cccd87-3d87-44f9-94d1-0a5bb2da237f None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:15.307349 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:15.317538 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application [None req-87cccd87-3d87-44f9-94d1-0a5bb2da237f None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 05 19:50:15.317538 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:50:15.317538 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:50:15.317538 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:50:15.317538 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:50:15.317538 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:50:15.317538 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:50:15.317538 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:50:15.317538 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:50:15.317538 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:50:15.317538 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:50:15.317538 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:50:15.317538 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 05 19:50:15.317538 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 05 19:50:15.317538 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 05 19:50:15.317538 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 05 19:50:15.318632 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:50:15.318632 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:50:15.318632 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 05 19:50:15.318632 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 19:50:15.318632 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 05 19:50:15.318632 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 19:50:15.318632 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:50:15.318632 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:50:15.318632 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:50:15.318632 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:50:15.318632 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:50:15.318632 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application with Lock( Aug 05 19:50:15.318632 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:50:15.318632 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:50:15.318632 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:50:15.318632 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:50:15.318632 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:50:15.318632 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:50:15.319766 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:50:15.319766 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:50:15.319766 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 05 19:50:15.319766 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 05 19:50:15.319766 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 05 19:50:15.319766 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 05 19:50:15.319766 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 05 19:50:15.319766 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 05 19:50:15.319766 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 05 19:50:15.319766 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Aug 05 19:50:15.322050 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 426/864] 10.222.0.27 () {56 vars in 1117 bytes} [Fri Aug 5 19:50:15 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:50:15.328605 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87cccd87-3d87-44f9-94d1-0a5bb2da237f None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:15.406972 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7803ce43-c803-488e-956d-3f2d5eea09dd None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:15.445395 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bbc8de3-dc70-4fd2-a5c4-23a8cb8c5f70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:15.446155 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbc8de3-dc70-4fd2-a5c4-23a8cb8c5f70 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:15.446549 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbc8de3-dc70-4fd2-a5c4-23a8cb8c5f70 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:15.446918 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbc8de3-dc70-4fd2-a5c4-23a8cb8c5f70 None admin] PATH_INFO: `/v3/projects` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:15.447773 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bbc8de3-dc70-4fd2-a5c4-23a8cb8c5f70 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:50:15.448181 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bbc8de3-dc70-4fd2-a5c4-23a8cb8c5f70 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:15.449739 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bbc8de3-dc70-4fd2-a5c4-23a8cb8c5f70 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:15.456772 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87cccd87-3d87-44f9-94d1-0a5bb2da237f None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:15.463592 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 439/865] 10.222.0.27 () {56 vars in 1148 bytes} [Fri Aug 5 19:50:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:50:15.470716 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bbc8de3-dc70-4fd2-a5c4-23a8cb8c5f70 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:15.504651 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a428ff68-8a07-41fe-b0ae-29d720e91fd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:15.505749 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a428ff68-8a07-41fe-b0ae-29d720e91fd8 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:15.506185 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a428ff68-8a07-41fe-b0ae-29d720e91fd8 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:15.506647 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a428ff68-8a07-41fe-b0ae-29d720e91fd8 None admin] PATH_INFO: `/v3/users` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:15.507856 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a428ff68-8a07-41fe-b0ae-29d720e91fd8 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:50:15.508315 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a428ff68-8a07-41fe-b0ae-29d720e91fd8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:15.510789 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a428ff68-8a07-41fe-b0ae-29d720e91fd8 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:15.541721 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 427/866] 10.222.0.27 () {56 vars in 1149 bytes} [Fri Aug 5 19:50:15 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 19:50:15.542942 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a428ff68-8a07-41fe-b0ae-29d720e91fd8 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:15.555151 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:15.589082 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bbc8de3-dc70-4fd2-a5c4-23a8cb8c5f70 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:15.620738 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:15.626687 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a2e379-af78-44a9-8c74-56046a8fbc96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:15.627482 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a2e379-af78-44a9-8c74-56046a8fbc96 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:15.627717 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a2e379-af78-44a9-8c74-56046a8fbc96 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:15.628139 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a2e379-af78-44a9-8c74-56046a8fbc96 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:15.629223 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a2e379-af78-44a9-8c74-56046a8fbc96 None admin] RBAC: Adding query filter params (role.id=c173688a960b46e5be65f0ccd6fbd734, scope.project.id=a70a82dcfc574fad90b3f5f00abbba58, user.id=6533931da2d34447881008b43372b0d5) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:50:15.629636 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a2e379-af78-44a9-8c74-56046a8fbc96 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:15.631284 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a2e379-af78-44a9-8c74-56046a8fbc96 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:15.641022 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 440/867] 10.222.0.27 () {56 vars in 1374 bytes} [Fri Aug 5 19:50:15 2022] GET /identity/v3/role_assignments?user.id=6533931da2d34447881008b43372b0d5&scope.project.id=a70a82dcfc574fad90b3f5f00abbba58&role.id=c173688a960b46e5be65f0ccd6fbd734 => generated 608 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:50:15.679046 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a428ff68-8a07-41fe-b0ae-29d720e91fd8 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:15.725884 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c757472a-9900-4f0b-89ed-80f0e3795095 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:15.726785 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c757472a-9900-4f0b-89ed-80f0e3795095 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:15.727214 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c757472a-9900-4f0b-89ed-80f0e3795095 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:15.727674 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c757472a-9900-4f0b-89ed-80f0e3795095 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:15.803423 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c757472a-9900-4f0b-89ed-80f0e3795095 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:15.805169 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c757472a-9900-4f0b-89ed-80f0e3795095 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:15.863358 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:15.917201 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 428/868] 10.222.0.27 () {58 vars in 1282 bytes} [Fri Aug 5 19:50:15 2022] GET /identity/v3/auth/tokens => generated 1262 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 05 19:50:15.923051 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a2e379-af78-44a9-8c74-56046a8fbc96 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:16.029424 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a2e379-af78-44a9-8c74-56046a8fbc96 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:16.064219 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a1cd0c-97b4-4a4b-ba30-0f0d99ff16e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:16.064930 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a1cd0c-97b4-4a4b-ba30-0f0d99ff16e5 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:16.065215 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a1cd0c-97b4-4a4b-ba30-0f0d99ff16e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:16.065484 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a1cd0c-97b4-4a4b-ba30-0f0d99ff16e5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:16.067409 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:16.077568 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a1cd0c-97b4-4a4b-ba30-0f0d99ff16e5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:16.079082 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a1cd0c-97b4-4a4b-ba30-0f0d99ff16e5 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:16.097892 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-f1a1cd0c-97b4-4a4b-ba30-0f0d99ff16e5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:50:16.097892 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:50:16.097892 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:50:16.097892 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:50:16.097892 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:50:16.097892 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:50:16.097892 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:50:16.097892 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:50:16.097892 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:50:16.097892 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:50:16.097892 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:50:16.097892 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:50:16.097892 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 19:50:16.097892 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 19:50:16.097892 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 19:50:16.098828 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 19:50:16.098828 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:50:16.098828 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:50:16.098828 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:50:16.098828 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:50:16.098828 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:50:16.098828 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:50:16.098828 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:50:16.098828 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Aug 05 19:50:16.098828 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:50:16.098828 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:50:16.098828 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:50:16.098828 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:50:16.098828 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:50:16.098828 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:50:16.098828 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:50:16.098828 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:50:16.099699 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 19:50:16.099699 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 19:50:16.099699 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:50:16.099699 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Aug 05 19:50:16.099699 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 441/869] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Aug 5 19:50:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 05 19:50:16.105011 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c757472a-9900-4f0b-89ed-80f0e3795095 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:16.166224 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcadaac-65c0-47be-9ff6-3f33566de137 None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:16.166565 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcadaac-65c0-47be-9ff6-3f33566de137 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:16.166783 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcadaac-65c0-47be-9ff6-3f33566de137 None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:16.168450 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 442/870] 10.222.0.27 () {54 vars in 927 bytes} [Fri Aug 5 19:50:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 05 19:50:16.173713 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad13a70-9ed4-4bd0-9ac3-130a26220208 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:16.174001 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad13a70-9ed4-4bd0-9ac3-130a26220208 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:16.174278 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad13a70-9ed4-4bd0-9ac3-130a26220208 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:16.195031 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-6ad13a70-9ed4-4bd0-9ac3-130a26220208 None None] MFA Rules not processed for user `03d1834a64cd4a43829827cb99a72747`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:50:16.196249 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-6ad13a70-9ed4-4bd0-9ac3-130a26220208 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:50:16.210884 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c757472a-9900-4f0b-89ed-80f0e3795095 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:16.261559 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67020038-d7cb-462b-a6f2-2a9c92a5fa0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:16.262333 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67020038-d7cb-462b-a6f2-2a9c92a5fa0a None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:16.262802 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67020038-d7cb-462b-a6f2-2a9c92a5fa0a None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:16.263169 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67020038-d7cb-462b-a6f2-2a9c92a5fa0a None admin] PATH_INFO: `/v3/domains` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:16.264514 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67020038-d7cb-462b-a6f2-2a9c92a5fa0a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:50:16.264927 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67020038-d7cb-462b-a6f2-2a9c92a5fa0a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:16.267001 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67020038-d7cb-462b-a6f2-2a9c92a5fa0a None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:16.278459 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 443/871] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Aug 5 19:50:16 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 05 19:50:16.288311 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 429/872] 10.222.0.27 () {56 vars in 1109 bytes} [Fri Aug 5 19:50:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:50:16.293707 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad13a70-9ed4-4bd0-9ac3-130a26220208 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:16.301558 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67020038-d7cb-462b-a6f2-2a9c92a5fa0a None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:16.320683 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:16.405350 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad13a70-9ed4-4bd0-9ac3-130a26220208 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:16.414800 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67020038-d7cb-462b-a6f2-2a9c92a5fa0a None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:16.424005 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:16.444582 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7227c037-ef2a-44e3-a184-827aa87951d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:16.445580 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7227c037-ef2a-44e3-a184-827aa87951d0 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:16.446084 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7227c037-ef2a-44e3-a184-827aa87951d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:16.446551 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7227c037-ef2a-44e3-a184-827aa87951d0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:16.452135 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8943838-3813-45c2-8264-086479f78995 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:16.452882 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8943838-3813-45c2-8264-086479f78995 None admin] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:16.453164 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8943838-3813-45c2-8264-086479f78995 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:16.453433 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8943838-3813-45c2-8264-086479f78995 None admin] PATH_INFO: `/v3/services` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:16.454140 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8943838-3813-45c2-8264-086479f78995 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:16.455507 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8943838-3813-45c2-8264-086479f78995 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:16.455822 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7227c037-ef2a-44e3-a184-827aa87951d0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:16.457475 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7227c037-ef2a-44e3-a184-827aa87951d0 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:16.472134 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-7227c037-ef2a-44e3-a184-827aa87951d0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 19:50:16.472134 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:50:16.472134 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:50:16.472134 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:50:16.472134 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:50:16.472134 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:50:16.472134 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:50:16.472134 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:50:16.472134 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:50:16.472134 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:50:16.472134 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:50:16.472134 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:50:16.472134 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 05 19:50:16.472134 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 05 19:50:16.472134 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 05 19:50:16.473204 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 05 19:50:16.473204 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:50:16.473204 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:50:16.473204 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:50:16.473204 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:50:16.473204 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:50:16.473204 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:50:16.473204 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:50:16.473204 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Aug 05 19:50:16.473204 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:50:16.473204 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:50:16.473204 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:50:16.473204 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:50:16.473204 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:50:16.473204 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:50:16.473204 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:50:16.473204 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:50:16.474370 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 05 19:50:16.474370 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 05 19:50:16.474370 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 05 19:50:16.474370 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 05 19:50:16.474370 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 05 19:50:16.474370 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 05 19:50:16.474370 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 19:50:16.474370 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Aug 05 19:50:16.476479 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 444/873] 10.222.0.27 () {56 vars in 1111 bytes} [Fri Aug 5 19:50:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:50:16.480642 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 430/874] 10.222.0.27 () {60 vars in 1141 bytes} [Fri Aug 5 19:50:16 2022] POST /identity/v3/services => generated 269 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 05 19:50:16.487112 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7227c037-ef2a-44e3-a184-827aa87951d0 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:16.602660 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7227c037-ef2a-44e3-a184-827aa87951d0 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:16.647133 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ddfd0e-7bca-4f81-ad12-5e6d6f2a314b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:16.647756 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ddfd0e-7bca-4f81-ad12-5e6d6f2a314b None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:16.648032 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ddfd0e-7bca-4f81-ad12-5e6d6f2a314b None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:16.648298 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ddfd0e-7bca-4f81-ad12-5e6d6f2a314b None admin] PATH_INFO: `/v3/projects` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:16.649094 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ddfd0e-7bca-4f81-ad12-5e6d6f2a314b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:50:16.649396 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ddfd0e-7bca-4f81-ad12-5e6d6f2a314b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:16.651145 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ddfd0e-7bca-4f81-ad12-5e6d6f2a314b None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:16.668686 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 445/875] 10.222.0.27 () {56 vars in 1148 bytes} [Fri Aug 5 19:50:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:50:16.672033 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8943838-3813-45c2-8264-086479f78995 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:16.763827 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:16.792813 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8943838-3813-45c2-8264-086479f78995 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:16.829630 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929d9a9d-5937-4725-984e-c2544d2759c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:16.830628 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929d9a9d-5937-4725-984e-c2544d2759c9 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:16.831110 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929d9a9d-5937-4725-984e-c2544d2759c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:16.831595 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929d9a9d-5937-4725-984e-c2544d2759c9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:16.832691 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-929d9a9d-5937-4725-984e-c2544d2759c9 None admin] RBAC: Adding query filter params (role.id=6e2bc1ba438d4eb69923ef4659c181f3, scope.project.id=a70a82dcfc574fad90b3f5f00abbba58, user.id=bdf7cf05a77e4298b2bf4f58b40f54aa) {{(pid=80364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:50:16.833224 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-929d9a9d-5937-4725-984e-c2544d2759c9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:16.834875 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-929d9a9d-5937-4725-984e-c2544d2759c9 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:16.841902 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 431/876] 10.222.0.27 () {56 vars in 1374 bytes} [Fri Aug 5 19:50:16 2022] GET /identity/v3/role_assignments?user.id=bdf7cf05a77e4298b2bf4f58b40f54aa&scope.project.id=a70a82dcfc574fad90b3f5f00abbba58&role.id=6e2bc1ba438d4eb69923ef4659c181f3 => generated 608 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 19:50:16.861175 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52864f51-03b5-4267-8a12-2060cf3ac066 None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:16.861509 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52864f51-03b5-4267-8a12-2060cf3ac066 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:16.861732 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52864f51-03b5-4267-8a12-2060cf3ac066 None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:16.863412 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 446/877] 10.222.0.27 () {54 vars in 927 bytes} [Fri Aug 5 19:50:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:50:16.870340 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63be2d8-4ccb-4ef3-bc31-6de1ad9dde26 None None] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:16.870657 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63be2d8-4ccb-4ef3-bc31-6de1ad9dde26 None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:16.870908 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63be2d8-4ccb-4ef3-bc31-6de1ad9dde26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:16.891162 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.auth.core [None req-a63be2d8-4ccb-4ef3-bc31-6de1ad9dde26 None None] MFA Rules not processed for user `03d1834a64cd4a43829827cb99a72747`. Rule list: `[]` (Enabled: `True`). {{(pid=80364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:50:16.892390 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-a63be2d8-4ccb-4ef3-bc31-6de1ad9dde26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:50:16.971750 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 432/878] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Aug 5 19:50:16 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 05 19:50:16.994949 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52864f51-03b5-4267-8a12-2060cf3ac066 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:17.031543 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:17.113865 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52864f51-03b5-4267-8a12-2060cf3ac066 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:17.149677 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12621c01-7be9-4cef-a8c3-11a2a48339a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:17.150621 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12621c01-7be9-4cef-a8c3-11a2a48339a7 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:17.150917 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12621c01-7be9-4cef-a8c3-11a2a48339a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:17.151194 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12621c01-7be9-4cef-a8c3-11a2a48339a7 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:17.151833 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12621c01-7be9-4cef-a8c3-11a2a48339a7 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:17.153763 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12621c01-7be9-4cef-a8c3-11a2a48339a7 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:17.163176 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-12621c01-7be9-4cef-a8c3-11a2a48339a7 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 05 19:50:17.163176 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:50:17.163176 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 05 19:50:17.163176 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 05 19:50:17.163176 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 05 19:50:17.163176 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 05 19:50:17.163176 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 05 19:50:17.163176 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 05 19:50:17.163176 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 05 19:50:17.163176 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Aug 05 19:50:17.163176 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 05 19:50:17.163176 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Aug 05 19:50:17.163176 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:50:17.163176 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:50:17.163176 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:50:17.163176 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:50:17.163176 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:50:17.164182 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:50:17.164182 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:50:17.164182 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:50:17.164182 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:50:17.164182 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:50:17.164182 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:50:17.164182 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 05 19:50:17.164182 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 05 19:50:17.164182 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 05 19:50:17.164182 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 05 19:50:17.164182 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:50:17.164182 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:50:17.164182 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:50:17.164182 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:50:17.164182 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:50:17.164182 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:50:17.165044 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:50:17.165044 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Aug 05 19:50:17.165044 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:50:17.165044 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:50:17.165044 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:50:17.165044 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:50:17.165044 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:50:17.165044 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:50:17.165044 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:50:17.165044 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:50:17.165044 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 05 19:50:17.165044 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 05 19:50:17.165044 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 05 19:50:17.165044 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Aug 05 19:50:17.165044 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 447/879] 10.222.0.27 () {56 vars in 1114 bytes} [Fri Aug 5 19:50:16 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:50:17.171159 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63be2d8-4ccb-4ef3-bc31-6de1ad9dde26 None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:17.241731 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:17.278090 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63be2d8-4ccb-4ef3-bc31-6de1ad9dde26 None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:17.316977 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489a0771-d311-4ef7-af54-594779517f71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:17.317518 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489a0771-d311-4ef7-af54-594779517f71 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:17.317800 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489a0771-d311-4ef7-af54-594779517f71 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:17.318075 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489a0771-d311-4ef7-af54-594779517f71 None admin] PATH_INFO: `/v3/services` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:17.318763 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489a0771-d311-4ef7-af54-594779517f71 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:50:17.319069 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489a0771-d311-4ef7-af54-594779517f71 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:17.320564 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489a0771-d311-4ef7-af54-594779517f71 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:17.325958 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 433/880] 10.222.0.27 () {56 vars in 1114 bytes} [Fri Aug 5 19:50:17 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:50:17.333627 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12621c01-7be9-4cef-a8c3-11a2a48339a7 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:17.436598 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:17.446710 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12621c01-7be9-4cef-a8c3-11a2a48339a7 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:17.492454 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d50d919-709a-4a9d-8e02-26ab0ae9727f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:17.493168 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d50d919-709a-4a9d-8e02-26ab0ae9727f None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:17.493451 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d50d919-709a-4a9d-8e02-26ab0ae9727f None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:17.493725 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d50d919-709a-4a9d-8e02-26ab0ae9727f None admin] PATH_INFO: `/v3/services` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:17.494460 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d50d919-709a-4a9d-8e02-26ab0ae9727f None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:50:17.494765 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d50d919-709a-4a9d-8e02-26ab0ae9727f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:17.496179 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d50d919-709a-4a9d-8e02-26ab0ae9727f None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:17.502395 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 448/881] 10.222.0.27 () {56 vars in 1114 bytes} [Fri Aug 5 19:50:17 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:50:17.508091 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489a0771-d311-4ef7-af54-594779517f71 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:17.616140 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489a0771-d311-4ef7-af54-594779517f71 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:17.647802 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:17.654279 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f2083d-026a-45ba-8d00-25bdd9bb8191 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:17.655221 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f2083d-026a-45ba-8d00-25bdd9bb8191 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:17.655839 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f2083d-026a-45ba-8d00-25bdd9bb8191 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:17.656332 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f2083d-026a-45ba-8d00-25bdd9bb8191 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:17.657379 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f2083d-026a-45ba-8d00-25bdd9bb8191 None admin] RBAC: Adding query filter params (interface=public, service_id=5ad6c1e77cb7449a8cd2f3db45033cc9, region_id=RegionOne) {{(pid=80364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:50:17.657907 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f2083d-026a-45ba-8d00-25bdd9bb8191 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:17.659489 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f2083d-026a-45ba-8d00-25bdd9bb8191 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:17.665484 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 434/882] 10.222.0.27 () {56 vars in 1251 bytes} [Fri Aug 5 19:50:17 2022] GET /identity/v3/endpoints?service_id=5ad6c1e77cb7449a8cd2f3db45033cc9&interface=public®ion_id=RegionOne => generated 194 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:50:17.676570 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d50d919-709a-4a9d-8e02-26ab0ae9727f None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:17.776885 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d50d919-709a-4a9d-8e02-26ab0ae9727f None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:17.812466 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae65faf6-c3a1-4acd-bf0a-1f4c96ab7f3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:17.813109 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae65faf6-c3a1-4acd-bf0a-1f4c96ab7f3f None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:17.813393 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae65faf6-c3a1-4acd-bf0a-1f4c96ab7f3f None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:17.813665 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae65faf6-c3a1-4acd-bf0a-1f4c96ab7f3f None admin] PATH_INFO: `/v3/services` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:17.814320 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae65faf6-c3a1-4acd-bf0a-1f4c96ab7f3f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:17.815667 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae65faf6-c3a1-4acd-bf0a-1f4c96ab7f3f None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:17.821703 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 449/883] 10.222.0.27 () {56 vars in 1087 bytes} [Fri Aug 5 19:50:17 2022] GET /identity/v3/services => generated 1268 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 19:50:17.898186 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48ef5c3-f3a7-430d-88c6-028f0b84db2b None None] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:17.898390 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48ef5c3-f3a7-430d-88c6-028f0b84db2b None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:17.898642 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48ef5c3-f3a7-430d-88c6-028f0b84db2b None None] PATH_INFO: `/` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:17.900619 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 435/884] 10.222.0.27 () {54 vars in 927 bytes} [Fri Aug 5 19:50:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:50:17.906839 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b6ce6b-d4cc-4ab3-9a43-76394656a33b None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:17.907168 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b6ce6b-d4cc-4ab3-9a43-76394656a33b None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:17.907400 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b6ce6b-d4cc-4ab3-9a43-76394656a33b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:17.927535 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-03b6ce6b-d4cc-4ab3-9a43-76394656a33b None None] MFA Rules not processed for user `03d1834a64cd4a43829827cb99a72747`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:50:17.928791 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-03b6ce6b-d4cc-4ab3-9a43-76394656a33b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:50:17.949060 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:18.003676 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 450/885] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Aug 5 19:50:17 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 05 19:50:18.025330 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48ef5c3-f3a7-430d-88c6-028f0b84db2b None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:18.128711 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48ef5c3-f3a7-430d-88c6-028f0b84db2b None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:18.163633 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7d729b-305e-4c14-b5bd-1cd3d535859b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:18.164897 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7d729b-305e-4c14-b5bd-1cd3d535859b None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:18.165182 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7d729b-305e-4c14-b5bd-1cd3d535859b None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:18.165450 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7d729b-305e-4c14-b5bd-1cd3d535859b None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:18.166101 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7d729b-305e-4c14-b5bd-1cd3d535859b None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:18.167524 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7d729b-305e-4c14-b5bd-1cd3d535859b None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:18.178270 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application [None req-4b7d729b-305e-4c14-b5bd-1cd3d535859b None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 05 19:50:18.178270 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:50:18.178270 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 05 19:50:18.178270 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 05 19:50:18.178270 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 05 19:50:18.178270 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 05 19:50:18.178270 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 05 19:50:18.178270 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 05 19:50:18.178270 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 05 19:50:18.178270 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Aug 05 19:50:18.178270 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 05 19:50:18.178270 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Aug 05 19:50:18.178270 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:50:18.178270 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:50:18.178270 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:50:18.178270 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:50:18.178270 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:50:18.179846 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:50:18.179846 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:50:18.179846 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:50:18.179846 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:50:18.179846 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:50:18.179846 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:50:18.179846 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 05 19:50:18.179846 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 05 19:50:18.179846 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 05 19:50:18.179846 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 05 19:50:18.179846 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:50:18.179846 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:50:18.179846 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:50:18.179846 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:50:18.179846 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:50:18.179846 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:50:18.180816 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:50:18.180816 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application with Lock( Aug 05 19:50:18.180816 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:50:18.180816 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:50:18.180816 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:50:18.180816 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:50:18.180816 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:50:18.180816 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:50:18.180816 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:50:18.180816 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:50:18.180816 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 05 19:50:18.180816 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 05 19:50:18.180816 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 05 19:50:18.180816 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Aug 05 19:50:18.180816 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 436/886] 10.222.0.27 () {56 vars in 1126 bytes} [Fri Aug 5 19:50:18 2022] GET /identity/v3/services/object-store => generated 93 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:50:18.186098 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b6ce6b-d4cc-4ab3-9a43-76394656a33b None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:18.290636 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b6ce6b-d4cc-4ab3-9a43-76394656a33b None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:18.326104 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2ae74e-c41a-4b66-a3c7-582e07a374ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:18.326731 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2ae74e-c41a-4b66-a3c7-582e07a374ac None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:18.327009 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2ae74e-c41a-4b66-a3c7-582e07a374ac None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:18.327286 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2ae74e-c41a-4b66-a3c7-582e07a374ac None admin] PATH_INFO: `/v3/services` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:18.327974 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd2ae74e-c41a-4b66-a3c7-582e07a374ac None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:50:18.328280 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd2ae74e-c41a-4b66-a3c7-582e07a374ac None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:18.329694 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd2ae74e-c41a-4b66-a3c7-582e07a374ac None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:18.334881 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 451/887] 10.222.0.27 () {56 vars in 1122 bytes} [Fri Aug 5 19:50:18 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:50:18.341306 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7d729b-305e-4c14-b5bd-1cd3d535859b None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:18.412193 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:18.455035 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7d729b-305e-4c14-b5bd-1cd3d535859b None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:18.489036 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032644ee-8102-4948-8c92-a38dd7182181 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:18.489940 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032644ee-8102-4948-8c92-a38dd7182181 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:18.490371 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032644ee-8102-4948-8c92-a38dd7182181 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:18.490833 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032644ee-8102-4948-8c92-a38dd7182181 None admin] PATH_INFO: `/v3/services` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:18.491710 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-032644ee-8102-4948-8c92-a38dd7182181 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:50:18.492327 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-032644ee-8102-4948-8c92-a38dd7182181 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:18.493866 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-032644ee-8102-4948-8c92-a38dd7182181 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:18.501203 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 437/888] 10.222.0.27 () {56 vars in 1122 bytes} [Fri Aug 5 19:50:18 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:50:18.757486 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de66fe4-4a40-437f-8011-e33056ec170e None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:18.757670 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de66fe4-4a40-437f-8011-e33056ec170e None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:18.757960 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de66fe4-4a40-437f-8011-e33056ec170e None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:18.759568 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 452/889] 10.222.0.27 () {54 vars in 927 bytes} [Fri Aug 5 19:50:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:50:18.764859 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd00e27-3012-4928-9e9b-7cc9191b5496 None None] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:18.765152 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd00e27-3012-4928-9e9b-7cc9191b5496 None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:18.765431 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd00e27-3012-4928-9e9b-7cc9191b5496 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:18.784489 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.auth.core [None req-9cd00e27-3012-4928-9e9b-7cc9191b5496 None None] MFA Rules not processed for user `03d1834a64cd4a43829827cb99a72747`. Rule list: `[]` (Enabled: `True`). {{(pid=80364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:50:18.785693 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-9cd00e27-3012-4928-9e9b-7cc9191b5496 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:50:18.860141 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 438/890] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Aug 5 19:50:18 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 05 19:50:18.886761 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de66fe4-4a40-437f-8011-e33056ec170e None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:18.911735 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:19.005900 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de66fe4-4a40-437f-8011-e33056ec170e None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:19.046079 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ffc7cf-3575-474a-869b-bfca09d3edff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:19.047094 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ffc7cf-3575-474a-869b-bfca09d3edff None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:19.047580 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ffc7cf-3575-474a-869b-bfca09d3edff None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:19.048095 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ffc7cf-3575-474a-869b-bfca09d3edff None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:19.048957 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ffc7cf-3575-474a-869b-bfca09d3edff None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:19.050595 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ffc7cf-3575-474a-869b-bfca09d3edff None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:19.060399 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-c1ffc7cf-3575-474a-869b-bfca09d3edff None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 05 19:50:19.060399 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:50:19.060399 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 05 19:50:19.060399 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 05 19:50:19.060399 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 05 19:50:19.060399 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 05 19:50:19.060399 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 05 19:50:19.060399 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 05 19:50:19.060399 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 05 19:50:19.060399 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Aug 05 19:50:19.060399 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 05 19:50:19.060399 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Aug 05 19:50:19.060399 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:50:19.060399 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:50:19.060399 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:50:19.060399 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:50:19.061361 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:50:19.061361 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:50:19.061361 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:50:19.061361 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:50:19.061361 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:50:19.061361 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:50:19.061361 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:50:19.061361 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 05 19:50:19.061361 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 05 19:50:19.061361 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 05 19:50:19.061361 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 05 19:50:19.061361 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:50:19.061361 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:50:19.061361 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:50:19.061361 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:50:19.061361 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:50:19.062194 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:50:19.062194 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:50:19.062194 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Aug 05 19:50:19.062194 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:50:19.062194 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:50:19.062194 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:50:19.062194 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:50:19.062194 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:50:19.062194 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:50:19.062194 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:50:19.062194 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:50:19.062194 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 05 19:50:19.062194 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 05 19:50:19.062194 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 05 19:50:19.062194 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Aug 05 19:50:19.065040 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 453/891] 10.222.0.27 () {56 vars in 1132 bytes} [Fri Aug 5 19:50:18 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:50:19.076562 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd00e27-3012-4928-9e9b-7cc9191b5496 None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:19.127682 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:19.170783 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23619b0-71b5-4f16-b9c5-0974bc4e8a81 None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:19.171130 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23619b0-71b5-4f16-b9c5-0974bc4e8a81 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:19.171338 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23619b0-71b5-4f16-b9c5-0974bc4e8a81 None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:19.173148 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 454/892] 10.222.0.27 () {54 vars in 927 bytes} [Fri Aug 5 19:50:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:50:19.177151 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd00e27-3012-4928-9e9b-7cc9191b5496 None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:19.178170 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dee82dd-4ecd-4aa1-a716-b3f4472595f8 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:19.178454 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dee82dd-4ecd-4aa1-a716-b3f4472595f8 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:19.178730 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dee82dd-4ecd-4aa1-a716-b3f4472595f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:19.200109 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-9dee82dd-4ecd-4aa1-a716-b3f4472595f8 None None] MFA Rules not processed for user `03d1834a64cd4a43829827cb99a72747`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:50:19.202302 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-9dee82dd-4ecd-4aa1-a716-b3f4472595f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:50:19.216709 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b5c7ac-bf16-44d0-bacd-c52855b6e3f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:19.217346 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b5c7ac-bf16-44d0-bacd-c52855b6e3f0 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:19.217829 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b5c7ac-bf16-44d0-bacd-c52855b6e3f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:19.218170 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b5c7ac-bf16-44d0-bacd-c52855b6e3f0 None admin] PATH_INFO: `/v3/services` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:19.218932 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b5c7ac-bf16-44d0-bacd-c52855b6e3f0 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:50:19.219392 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b5c7ac-bf16-44d0-bacd-c52855b6e3f0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:19.220882 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b5c7ac-bf16-44d0-bacd-c52855b6e3f0 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:19.226742 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 439/893] 10.222.0.27 () {56 vars in 1126 bytes} [Fri Aug 5 19:50:19 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:50:19.233847 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b5c7ac-bf16-44d0-bacd-c52855b6e3f0 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:19.288760 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 455/894] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Aug 5 19:50:19 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 05 19:50:19.304370 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dee82dd-4ecd-4aa1-a716-b3f4472595f8 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:19.319655 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:19.346063 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b5c7ac-bf16-44d0-bacd-c52855b6e3f0 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:19.374970 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:19.395600 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21df8a4-8410-40e8-accf-e07a56d2dfcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:19.396705 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21df8a4-8410-40e8-accf-e07a56d2dfcf None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:19.397189 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21df8a4-8410-40e8-accf-e07a56d2dfcf None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:19.397813 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21df8a4-8410-40e8-accf-e07a56d2dfcf None admin] PATH_INFO: `/v3/services` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:19.398727 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e21df8a4-8410-40e8-accf-e07a56d2dfcf None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:50:19.399215 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e21df8a4-8410-40e8-accf-e07a56d2dfcf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:19.401249 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e21df8a4-8410-40e8-accf-e07a56d2dfcf None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:19.413266 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 440/895] 10.222.0.27 () {56 vars in 1126 bytes} [Fri Aug 5 19:50:19 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:50:19.413971 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dee82dd-4ecd-4aa1-a716-b3f4472595f8 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:19.414771 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21df8a4-8410-40e8-accf-e07a56d2dfcf None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:19.453383 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d908167-ea8d-409b-9600-f1b45ad07bd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:19.454105 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d908167-ea8d-409b-9600-f1b45ad07bd7 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:19.454462 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d908167-ea8d-409b-9600-f1b45ad07bd7 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:19.455539 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d908167-ea8d-409b-9600-f1b45ad07bd7 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:19.467027 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d908167-ea8d-409b-9600-f1b45ad07bd7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 19:50:19.472542 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-5d908167-ea8d-409b-9600-f1b45ad07bd7 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 05 19:50:19.472542 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:50:19.472542 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:50:19.472542 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:50:19.472542 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:50:19.472542 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:50:19.472542 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:50:19.472542 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:50:19.472542 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:50:19.472542 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:50:19.472542 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:50:19.472542 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:50:19.472542 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 19:50:19.472542 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 19:50:19.472542 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 19:50:19.472542 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 19:50:19.473751 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 19:50:19.473751 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 19:50:19.473751 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 19:50:19.473751 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 19:50:19.473751 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:50:19.473751 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:50:19.473751 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:50:19.473751 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:50:19.473751 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:50:19.473751 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:50:19.473751 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:50:19.473751 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Aug 05 19:50:19.473751 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:50:19.473751 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:50:19.473751 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:50:19.473751 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:50:19.474805 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:50:19.474805 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:50:19.474805 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:50:19.474805 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:50:19.474805 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 19:50:19.474805 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 19:50:19.474805 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 19:50:19.474805 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 19:50:19.474805 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 19:50:19.474805 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 19:50:19.474805 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 05 19:50:19.474805 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Aug 05 19:50:19.480791 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 456/896] 10.222.0.27 () {56 vars in 1120 bytes} [Fri Aug 5 19:50:19 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 05 19:50:19.483002 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d908167-ea8d-409b-9600-f1b45ad07bd7 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:19.534032 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21df8a4-8410-40e8-accf-e07a56d2dfcf None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:19.563706 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:19.576969 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2538443-980f-46b2-8ca4-1cd78c7f136e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:19.577871 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2538443-980f-46b2-8ca4-1cd78c7f136e None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:19.578317 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2538443-980f-46b2-8ca4-1cd78c7f136e None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:19.578803 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2538443-980f-46b2-8ca4-1cd78c7f136e None admin] PATH_INFO: `/v3/endpoints` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:19.579825 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2538443-980f-46b2-8ca4-1cd78c7f136e None admin] RBAC: Adding query filter params (interface=public, service_id=d8510b5a25aa403c839050f92baed0d7, region_id=RegionOne) {{(pid=80364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:50:19.580307 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2538443-980f-46b2-8ca4-1cd78c7f136e None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:19.581965 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2538443-980f-46b2-8ca4-1cd78c7f136e None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:19.593002 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 441/897] 10.222.0.27 () {56 vars in 1251 bytes} [Fri Aug 5 19:50:19 2022] GET /identity/v3/endpoints?service_id=d8510b5a25aa403c839050f92baed0d7&interface=public®ion_id=RegionOne => generated 194 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:50:19.594349 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2538443-980f-46b2-8ca4-1cd78c7f136e None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:19.601707 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d908167-ea8d-409b-9600-f1b45ad07bd7 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:19.644047 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4817cf-652e-4c66-8ac4-6e1ec4dd75c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:19.644827 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4817cf-652e-4c66-8ac4-6e1ec4dd75c1 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:19.645190 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4817cf-652e-4c66-8ac4-6e1ec4dd75c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:19.645690 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4817cf-652e-4c66-8ac4-6e1ec4dd75c1 None admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:19.646515 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4817cf-652e-4c66-8ac4-6e1ec4dd75c1 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:50:19.646900 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4817cf-652e-4c66-8ac4-6e1ec4dd75c1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:19.648786 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4817cf-652e-4c66-8ac4-6e1ec4dd75c1 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:19.657712 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:19.665900 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 457/898] 10.222.0.27 () {56 vars in 1115 bytes} [Fri Aug 5 19:50:19 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 19:50:19.667864 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4817cf-652e-4c66-8ac4-6e1ec4dd75c1 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:19.752395 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2538443-980f-46b2-8ca4-1cd78c7f136e None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:19.789065 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4817cf-652e-4c66-8ac4-6e1ec4dd75c1 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:19.803589 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283ba30c-d647-4721-a333-307c7c6cac81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:19.804891 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283ba30c-d647-4721-a333-307c7c6cac81 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:19.805777 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283ba30c-d647-4721-a333-307c7c6cac81 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:19.806027 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283ba30c-d647-4721-a333-307c7c6cac81 None admin] PATH_INFO: `/v3/services` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:19.807317 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-283ba30c-d647-4721-a333-307c7c6cac81 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:19.810121 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-283ba30c-d647-4721-a333-307c7c6cac81 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:19.822716 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 442/899] 10.222.0.27 () {56 vars in 1087 bytes} [Fri Aug 5 19:50:19 2022] GET /identity/v3/services => generated 1268 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 19:50:19.830807 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72c6ae0-60a6-4c2f-9817-7fe61f14f289 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:19.832316 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72c6ae0-60a6-4c2f-9817-7fe61f14f289 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:19.832862 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72c6ae0-60a6-4c2f-9817-7fe61f14f289 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:19.833323 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72c6ae0-60a6-4c2f-9817-7fe61f14f289 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:19.846301 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72c6ae0-60a6-4c2f-9817-7fe61f14f289 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:19.848148 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72c6ae0-60a6-4c2f-9817-7fe61f14f289 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:19.857103 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:19.867068 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-b72c6ae0-60a6-4c2f-9817-7fe61f14f289 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:50:19.867068 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:50:19.867068 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:50:19.867068 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:50:19.867068 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:50:19.867068 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:50:19.867068 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:50:19.867068 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:50:19.867068 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:50:19.867068 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:50:19.867068 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:50:19.867068 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:50:19.867068 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 19:50:19.867068 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 19:50:19.867068 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 19:50:19.868056 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 19:50:19.868056 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:50:19.868056 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:50:19.868056 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:50:19.868056 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:50:19.868056 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:50:19.868056 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:50:19.868056 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:50:19.868056 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Aug 05 19:50:19.868056 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:50:19.868056 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:50:19.868056 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:50:19.868056 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:50:19.868056 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:50:19.868056 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:50:19.868056 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:50:19.868056 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:50:19.869010 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 19:50:19.869010 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 19:50:19.869010 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:50:19.869010 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Aug 05 19:50:19.870891 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 458/900] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Aug 5 19:50:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:50:19.881683 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283ba30c-d647-4721-a333-307c7c6cac81 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:20.074151 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283ba30c-d647-4721-a333-307c7c6cac81 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:20.093481 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:20.100483 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0750bc0e-bd38-442e-95ea-2acf142f12bd None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:20.100897 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0750bc0e-bd38-442e-95ea-2acf142f12bd None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:20.101271 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0750bc0e-bd38-442e-95ea-2acf142f12bd None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:20.103062 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 459/901] 10.222.0.27 () {54 vars in 927 bytes} [Fri Aug 5 19:50:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:50:20.109101 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f2aa4d-a2b7-4353-ab96-ee977dca2a35 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:20.109442 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f2aa4d-a2b7-4353-ab96-ee977dca2a35 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:20.109784 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f2aa4d-a2b7-4353-ab96-ee977dca2a35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:20.127851 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cebc4b5-b547-47df-841b-b021ea7741a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:20.128484 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cebc4b5-b547-47df-841b-b021ea7741a4 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:20.128856 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cebc4b5-b547-47df-841b-b021ea7741a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:20.129139 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cebc4b5-b547-47df-841b-b021ea7741a4 None admin] PATH_INFO: `/v3/domains` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:20.129920 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cebc4b5-b547-47df-841b-b021ea7741a4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:50:20.130226 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cebc4b5-b547-47df-841b-b021ea7741a4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:20.131678 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cebc4b5-b547-47df-841b-b021ea7741a4 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:20.134530 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-02f2aa4d-a2b7-4353-ab96-ee977dca2a35 None None] MFA Rules not processed for user `03d1834a64cd4a43829827cb99a72747`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:50:20.136767 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-02f2aa4d-a2b7-4353-ab96-ee977dca2a35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:50:20.154751 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 443/902] 10.222.0.27 () {56 vars in 1109 bytes} [Fri Aug 5 19:50:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 19:50:20.160442 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cebc4b5-b547-47df-841b-b021ea7741a4 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:20.244649 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 460/903] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Aug 5 19:50:20 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 05 19:50:20.269259 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f2aa4d-a2b7-4353-ab96-ee977dca2a35 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:20.316163 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cebc4b5-b547-47df-841b-b021ea7741a4 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:20.369170 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61bd3b99-e37d-4d32-b7f8-b0e59480414b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:20.369170 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bd3b99-e37d-4d32-b7f8-b0e59480414b None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:20.369170 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bd3b99-e37d-4d32-b7f8-b0e59480414b None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:20.369170 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bd3b99-e37d-4d32-b7f8-b0e59480414b None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:20.380301 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61bd3b99-e37d-4d32-b7f8-b0e59480414b None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:20.381993 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61bd3b99-e37d-4d32-b7f8-b0e59480414b None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:20.392757 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f2aa4d-a2b7-4353-ab96-ee977dca2a35 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:20.397674 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application [None req-61bd3b99-e37d-4d32-b7f8-b0e59480414b None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 05 19:50:20.397674 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:50:20.397674 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:50:20.397674 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:50:20.397674 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:50:20.397674 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:50:20.397674 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:50:20.397674 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:50:20.397674 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:50:20.397674 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:50:20.397674 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:50:20.397674 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:50:20.397674 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 05 19:50:20.397674 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 05 19:50:20.397674 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 05 19:50:20.397674 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 05 19:50:20.398625 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:50:20.398625 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:50:20.398625 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 05 19:50:20.398625 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 19:50:20.398625 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 05 19:50:20.398625 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 19:50:20.398625 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:50:20.398625 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:50:20.398625 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:50:20.398625 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:50:20.398625 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:50:20.398625 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application with Lock( Aug 05 19:50:20.398625 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:50:20.398625 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:50:20.398625 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:50:20.398625 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:50:20.398625 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:50:20.398625 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:50:20.399847 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:50:20.399847 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:50:20.399847 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 05 19:50:20.399847 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 05 19:50:20.399847 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 05 19:50:20.399847 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 05 19:50:20.399847 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 05 19:50:20.399847 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 05 19:50:20.399847 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 05 19:50:20.399847 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Aug 05 19:50:20.404440 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 444/904] 10.222.0.27 () {56 vars in 1117 bytes} [Fri Aug 5 19:50:20 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 05 19:50:20.405899 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61bd3b99-e37d-4d32-b7f8-b0e59480414b None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:20.416436 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:20.439432 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a6e9c2-58bb-4e0d-9777-cbacbd0e4966 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:20.440621 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a6e9c2-58bb-4e0d-9777-cbacbd0e4966 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:20.441173 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a6e9c2-58bb-4e0d-9777-cbacbd0e4966 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:20.441651 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a6e9c2-58bb-4e0d-9777-cbacbd0e4966 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:20.442499 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a6e9c2-58bb-4e0d-9777-cbacbd0e4966 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:20.443158 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:20.445854 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a6e9c2-58bb-4e0d-9777-cbacbd0e4966 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:20.457127 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-51a6e9c2-58bb-4e0d-9777-cbacbd0e4966 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 05 19:50:20.457127 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:50:20.457127 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 05 19:50:20.457127 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 05 19:50:20.457127 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 05 19:50:20.457127 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 05 19:50:20.457127 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 05 19:50:20.457127 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 05 19:50:20.457127 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 05 19:50:20.457127 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Aug 05 19:50:20.457127 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 05 19:50:20.457127 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Aug 05 19:50:20.457127 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:50:20.457127 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:50:20.457127 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:50:20.457127 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:50:20.457127 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:50:20.458196 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:50:20.458196 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:50:20.458196 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:50:20.458196 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:50:20.458196 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:50:20.458196 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:50:20.458196 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 05 19:50:20.458196 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 05 19:50:20.458196 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 05 19:50:20.458196 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 05 19:50:20.458196 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:50:20.458196 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:50:20.458196 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:50:20.458196 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:50:20.458196 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:50:20.458196 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:50:20.459051 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:50:20.459051 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Aug 05 19:50:20.459051 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:50:20.459051 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:50:20.459051 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:50:20.459051 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:50:20.459051 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:50:20.459051 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:50:20.459051 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:50:20.459051 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:50:20.459051 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 05 19:50:20.459051 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 05 19:50:20.459051 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 05 19:50:20.459051 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Aug 05 19:50:20.467518 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 461/905] 10.222.0.27 () {56 vars in 1114 bytes} [Fri Aug 5 19:50:20 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:50:20.472052 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a6e9c2-58bb-4e0d-9777-cbacbd0e4966 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:20.519647 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61bd3b99-e37d-4d32-b7f8-b0e59480414b None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:20.556751 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17cf6e82-6403-4af7-bd0c-a38d88402c39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:20.557391 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17cf6e82-6403-4af7-bd0c-a38d88402c39 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:20.557671 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17cf6e82-6403-4af7-bd0c-a38d88402c39 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:20.557947 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17cf6e82-6403-4af7-bd0c-a38d88402c39 None admin] PATH_INFO: `/v3/users` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:20.558870 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17cf6e82-6403-4af7-bd0c-a38d88402c39 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:50:20.559182 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17cf6e82-6403-4af7-bd0c-a38d88402c39 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:20.560913 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17cf6e82-6403-4af7-bd0c-a38d88402c39 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:20.581164 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 445/906] 10.222.0.27 () {56 vars in 1149 bytes} [Fri Aug 5 19:50:20 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:50:20.582471 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16426011-ad1b-4757-a4c3-20c7d650487b None None] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:20.582755 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16426011-ad1b-4757-a4c3-20c7d650487b None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:20.583028 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16426011-ad1b-4757-a4c3-20c7d650487b None None] PATH_INFO: `/` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:20.584419 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 446/907] 10.222.0.27 () {54 vars in 927 bytes} [Fri Aug 5 19:50:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:50:20.589787 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16426011-ad1b-4757-a4c3-20c7d650487b None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:20.615231 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a6e9c2-58bb-4e0d-9777-cbacbd0e4966 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:20.655519 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5558e3fd-b1c3-4ecd-99fd-7c5396b3d17a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:20.656244 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5558e3fd-b1c3-4ecd-99fd-7c5396b3d17a None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:20.656681 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5558e3fd-b1c3-4ecd-99fd-7c5396b3d17a None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:20.657089 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5558e3fd-b1c3-4ecd-99fd-7c5396b3d17a None admin] PATH_INFO: `/v3/services` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:20.658148 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5558e3fd-b1c3-4ecd-99fd-7c5396b3d17a None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:50:20.658566 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5558e3fd-b1c3-4ecd-99fd-7c5396b3d17a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:20.660266 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5558e3fd-b1c3-4ecd-99fd-7c5396b3d17a None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:20.667206 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 462/908] 10.222.0.27 () {56 vars in 1114 bytes} [Fri Aug 5 19:50:20 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:50:20.669936 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa624d1-7cc3-413c-8a73-4d8f205b97dd None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:20.670416 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa624d1-7cc3-413c-8a73-4d8f205b97dd None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:20.670888 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa624d1-7cc3-413c-8a73-4d8f205b97dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:20.691754 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-bfa624d1-7cc3-413c-8a73-4d8f205b97dd None None] MFA Rules not processed for user `03d1834a64cd4a43829827cb99a72747`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:50:20.693006 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-bfa624d1-7cc3-413c-8a73-4d8f205b97dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:50:20.706231 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:20.722166 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16426011-ad1b-4757-a4c3-20c7d650487b None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:20.740708 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:20.760870 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4323f028-5d89-4147-ba2c-e1c3194e73f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:20.761476 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4323f028-5d89-4147-ba2c-e1c3194e73f5 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:20.761764 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4323f028-5d89-4147-ba2c-e1c3194e73f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:20.762038 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4323f028-5d89-4147-ba2c-e1c3194e73f5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:20.780642 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 463/909] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Aug 5 19:50:20 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 05 19:50:20.781518 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa624d1-7cc3-413c-8a73-4d8f205b97dd None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:20.833643 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4323f028-5d89-4147-ba2c-e1c3194e73f5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:20.835185 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4323f028-5d89-4147-ba2c-e1c3194e73f5 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:20.886591 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa624d1-7cc3-413c-8a73-4d8f205b97dd None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:20.921470 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604789de-dc81-4433-8d9a-9507d4567912 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:20.922100 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604789de-dc81-4433-8d9a-9507d4567912 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:20.922383 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604789de-dc81-4433-8d9a-9507d4567912 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:20.922660 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604789de-dc81-4433-8d9a-9507d4567912 None admin] PATH_INFO: `/v3/services` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:20.923335 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-604789de-dc81-4433-8d9a-9507d4567912 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:50:20.923700 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-604789de-dc81-4433-8d9a-9507d4567912 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:20.925113 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-604789de-dc81-4433-8d9a-9507d4567912 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:20.925657 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:20.930466 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 464/910] 10.222.0.27 () {56 vars in 1114 bytes} [Fri Aug 5 19:50:20 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:50:20.932773 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604789de-dc81-4433-8d9a-9507d4567912 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:20.945506 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 447/911] 10.222.0.27 () {58 vars in 1282 bytes} [Fri Aug 5 19:50:20 2022] GET /identity/v3/auth/tokens => generated 1372 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 05 19:50:20.950662 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4323f028-5d89-4147-ba2c-e1c3194e73f5 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:20.973683 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:21.039997 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604789de-dc81-4433-8d9a-9507d4567912 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:21.061576 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4323f028-5d89-4147-ba2c-e1c3194e73f5 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:21.077884 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a753b0-de53-45fa-aa28-7de617f6f193 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:21.078615 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a753b0-de53-45fa-aa28-7de617f6f193 None admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:21.078896 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a753b0-de53-45fa-aa28-7de617f6f193 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:21.079171 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a753b0-de53-45fa-aa28-7de617f6f193 None admin] PATH_INFO: `/v3/services` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:21.079870 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a753b0-de53-45fa-aa28-7de617f6f193 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:21.081393 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a753b0-de53-45fa-aa28-7de617f6f193 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:21.095971 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16a1bbb-0547-4015-b939-2f62a4e4d216 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:21.096614 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16a1bbb-0547-4015-b939-2f62a4e4d216 None admin] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:21.097078 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16a1bbb-0547-4015-b939-2f62a4e4d216 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:21.097369 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16a1bbb-0547-4015-b939-2f62a4e4d216 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:21.098067 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e16a1bbb-0547-4015-b939-2f62a4e4d216 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:21.099444 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e16a1bbb-0547-4015-b939-2f62a4e4d216 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:21.100659 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 465/912] 10.222.0.27 () {60 vars in 1141 bytes} [Fri Aug 5 19:50:20 2022] POST /identity/v3/services => generated 241 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 19:50:21.101943 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a753b0-de53-45fa-aa28-7de617f6f193 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:21.137244 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 448/913] 10.222.0.27 () {60 vars in 1144 bytes} [Fri Aug 5 19:50:20 2022] POST /identity/v3/endpoints => generated 346 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 19:50:21.230902 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:21.243040 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a753b0-de53-45fa-aa28-7de617f6f193 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:21.296430 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3bded41-34ab-476c-943d-5382b4317460 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:21.297642 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bded41-34ab-476c-943d-5382b4317460 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:21.298128 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bded41-34ab-476c-943d-5382b4317460 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:21.298451 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bded41-34ab-476c-943d-5382b4317460 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:21.312595 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3bded41-34ab-476c-943d-5382b4317460 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:21.314139 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3bded41-34ab-476c-943d-5382b4317460 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:21.332591 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-f3bded41-34ab-476c-943d-5382b4317460 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:50:21.332591 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:50:21.332591 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:50:21.332591 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:50:21.332591 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:50:21.332591 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:50:21.332591 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:50:21.332591 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:50:21.332591 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:50:21.332591 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:50:21.332591 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:50:21.332591 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:50:21.332591 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 19:50:21.332591 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 19:50:21.332591 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 19:50:21.333738 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 19:50:21.333738 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:50:21.333738 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:50:21.333738 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:50:21.333738 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:50:21.333738 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:50:21.333738 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:50:21.333738 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:50:21.333738 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Aug 05 19:50:21.333738 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:50:21.333738 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:50:21.333738 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:50:21.333738 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:50:21.333738 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:50:21.333738 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:50:21.333738 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:50:21.333738 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:50:21.334843 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 19:50:21.334843 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 19:50:21.334843 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:50:21.334843 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Aug 05 19:50:21.334843 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 466/914] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Aug 5 19:50:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:50:21.340408 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16a1bbb-0547-4015-b939-2f62a4e4d216 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:21.413701 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:21.490864 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16a1bbb-0547-4015-b939-2f62a4e4d216 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:21.528995 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26773e3d-bc24-4eab-9105-66ca991004f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:21.529631 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26773e3d-bc24-4eab-9105-66ca991004f1 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:21.529915 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26773e3d-bc24-4eab-9105-66ca991004f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:21.530193 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26773e3d-bc24-4eab-9105-66ca991004f1 None admin] PATH_INFO: `/v3/domains` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:21.530904 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26773e3d-bc24-4eab-9105-66ca991004f1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:50:21.531209 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26773e3d-bc24-4eab-9105-66ca991004f1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:21.532636 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26773e3d-bc24-4eab-9105-66ca991004f1 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:21.550771 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 449/915] 10.222.0.27 () {56 vars in 1109 bytes} [Fri Aug 5 19:50:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:50:21.556585 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3bded41-34ab-476c-943d-5382b4317460 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:21.684399 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3bded41-34ab-476c-943d-5382b4317460 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:21.734938 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63260b7c-0522-4075-b544-ac037818cac3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:21.736696 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63260b7c-0522-4075-b544-ac037818cac3 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:21.736696 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63260b7c-0522-4075-b544-ac037818cac3 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:21.736696 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63260b7c-0522-4075-b544-ac037818cac3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:21.744172 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63260b7c-0522-4075-b544-ac037818cac3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:21.746042 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63260b7c-0522-4075-b544-ac037818cac3 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:21.758323 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-63260b7c-0522-4075-b544-ac037818cac3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 19:50:21.758323 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:50:21.758323 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:50:21.758323 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:50:21.758323 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:50:21.758323 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:50:21.758323 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:50:21.758323 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:50:21.758323 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:50:21.758323 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:50:21.758323 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:50:21.758323 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:50:21.758323 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 05 19:50:21.758323 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 05 19:50:21.758323 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 05 19:50:21.759282 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 05 19:50:21.759282 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:50:21.759282 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:50:21.759282 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:50:21.759282 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:50:21.759282 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:50:21.759282 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:50:21.759282 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:50:21.759282 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Aug 05 19:50:21.759282 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:50:21.759282 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:50:21.759282 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:50:21.759282 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:50:21.759282 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:50:21.759282 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:50:21.759282 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:50:21.759282 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:50:21.760154 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 05 19:50:21.760154 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 05 19:50:21.760154 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 05 19:50:21.760154 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 05 19:50:21.760154 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 05 19:50:21.760154 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 05 19:50:21.760154 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 19:50:21.760154 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Aug 05 19:50:21.762678 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 467/916] 10.222.0.27 () {56 vars in 1111 bytes} [Fri Aug 5 19:50:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:50:21.769816 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26773e3d-bc24-4eab-9105-66ca991004f1 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:21.912637 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26773e3d-bc24-4eab-9105-66ca991004f1 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:21.915350 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:21.961578 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2fbf16-542a-414c-aadc-d79e41b06533 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:21.962554 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2fbf16-542a-414c-aadc-d79e41b06533 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:21.963048 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2fbf16-542a-414c-aadc-d79e41b06533 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:21.963516 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2fbf16-542a-414c-aadc-d79e41b06533 None admin] PATH_INFO: `/v3/projects` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:21.965007 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d2fbf16-542a-414c-aadc-d79e41b06533 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:50:21.966125 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d2fbf16-542a-414c-aadc-d79e41b06533 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:21.967680 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d2fbf16-542a-414c-aadc-d79e41b06533 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:21.982595 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 450/917] 10.222.0.27 () {56 vars in 1148 bytes} [Fri Aug 5 19:50:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:50:21.992488 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63260b7c-0522-4075-b544-ac037818cac3 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:21.998050 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:22.035671 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bfe5bf-cb60-499b-930a-c5fe6397428b None None] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:22.035971 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bfe5bf-cb60-499b-930a-c5fe6397428b None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:22.036280 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bfe5bf-cb60-499b-930a-c5fe6397428b None None] PATH_INFO: `/` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:22.037940 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 451/918] 10.222.0.27 () {54 vars in 927 bytes} [Fri Aug 5 19:50:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:50:22.043501 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c6fcd1-54b2-43ef-bb3a-52ad037806dc None None] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:22.043907 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c6fcd1-54b2-43ef-bb3a-52ad037806dc None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:22.044112 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c6fcd1-54b2-43ef-bb3a-52ad037806dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:22.063530 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.auth.core [None req-54c6fcd1-54b2-43ef-bb3a-52ad037806dc None None] MFA Rules not processed for user `03d1834a64cd4a43829827cb99a72747`. Rule list: `[]` (Enabled: `True`). {{(pid=80364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:50:22.064799 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-54c6fcd1-54b2-43ef-bb3a-52ad037806dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:50:22.150936 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63260b7c-0522-4075-b544-ac037818cac3 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:22.164652 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 452/919] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Aug 5 19:50:22 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 05 19:50:22.184957 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c6fcd1-54b2-43ef-bb3a-52ad037806dc None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:22.195294 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d23269b-99ec-4558-841c-7fc3eabc68e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:22.195906 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d23269b-99ec-4558-841c-7fc3eabc68e8 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:22.196198 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d23269b-99ec-4558-841c-7fc3eabc68e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:22.196471 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d23269b-99ec-4558-841c-7fc3eabc68e8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:22.197317 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d23269b-99ec-4558-841c-7fc3eabc68e8 None admin] RBAC: Adding query filter params (role.id=31647643dd7c43aba270f63d5fe83acb, scope.project.id=a70a82dcfc574fad90b3f5f00abbba58, user.id=bdf7cf05a77e4298b2bf4f58b40f54aa) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:50:22.197648 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d23269b-99ec-4558-841c-7fc3eabc68e8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:22.199183 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d23269b-99ec-4558-841c-7fc3eabc68e8 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:22.212111 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 468/920] 10.222.0.27 () {56 vars in 1374 bytes} [Fri Aug 5 19:50:21 2022] GET /identity/v3/role_assignments?user.id=bdf7cf05a77e4298b2bf4f58b40f54aa&scope.project.id=a70a82dcfc574fad90b3f5f00abbba58&role.id=31647643dd7c43aba270f63d5fe83acb => generated 259 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:50:22.302028 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:22.309604 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c6fcd1-54b2-43ef-bb3a-52ad037806dc None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:22.348286 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e34dce6-c36c-4bc0-accf-d3e791a9a90d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:22.349087 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e34dce6-c36c-4bc0-accf-d3e791a9a90d None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:22.349250 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e34dce6-c36c-4bc0-accf-d3e791a9a90d None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:22.349519 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e34dce6-c36c-4bc0-accf-d3e791a9a90d None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:22.350166 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e34dce6-c36c-4bc0-accf-d3e791a9a90d None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:22.351604 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e34dce6-c36c-4bc0-accf-d3e791a9a90d None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:22.364757 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application [None req-6e34dce6-c36c-4bc0-accf-d3e791a9a90d None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 05 19:50:22.364757 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:50:22.364757 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 05 19:50:22.364757 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 05 19:50:22.364757 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 05 19:50:22.364757 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 05 19:50:22.364757 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 05 19:50:22.364757 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 05 19:50:22.364757 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 05 19:50:22.364757 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Aug 05 19:50:22.364757 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 05 19:50:22.364757 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Aug 05 19:50:22.364757 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:50:22.364757 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:50:22.364757 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:50:22.364757 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:50:22.365749 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:50:22.365749 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:50:22.365749 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:50:22.365749 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:50:22.365749 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:50:22.365749 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:50:22.365749 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:50:22.365749 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 05 19:50:22.365749 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 05 19:50:22.365749 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 05 19:50:22.365749 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 05 19:50:22.365749 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:50:22.365749 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:50:22.365749 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:50:22.365749 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:50:22.365749 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:50:22.367029 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:50:22.367029 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:50:22.367029 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application with Lock( Aug 05 19:50:22.367029 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:50:22.367029 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:50:22.367029 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:50:22.367029 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:50:22.367029 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:50:22.367029 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:50:22.367029 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:50:22.367029 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:50:22.367029 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 05 19:50:22.367029 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 05 19:50:22.367029 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 05 19:50:22.367029 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Aug 05 19:50:22.367029 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 453/921] 10.222.0.27 () {56 vars in 1132 bytes} [Fri Aug 5 19:50:22 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 05 19:50:22.372666 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d23269b-99ec-4558-841c-7fc3eabc68e8 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:22.445558 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:22.484547 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d23269b-99ec-4558-841c-7fc3eabc68e8 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:22.524702 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22b2de3-b0d4-4daf-bd35-98f66ca0be6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:22.525332 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22b2de3-b0d4-4daf-bd35-98f66ca0be6d None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:22.525612 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22b2de3-b0d4-4daf-bd35-98f66ca0be6d None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:22.525944 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22b2de3-b0d4-4daf-bd35-98f66ca0be6d None admin] PATH_INFO: `/v3/services` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:22.526692 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22b2de3-b0d4-4daf-bd35-98f66ca0be6d None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:50:22.527058 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22b2de3-b0d4-4daf-bd35-98f66ca0be6d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:22.529023 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22b2de3-b0d4-4daf-bd35-98f66ca0be6d None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:22.534418 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 469/922] 10.222.0.27 () {56 vars in 1126 bytes} [Fri Aug 5 19:50:22 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:50:22.539819 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e34dce6-c36c-4bc0-accf-d3e791a9a90d None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:22.652863 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e34dce6-c36c-4bc0-accf-d3e791a9a90d None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:22.689747 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021a1f42-9213-4474-9fed-5ad275d4e028 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:22.690723 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021a1f42-9213-4474-9fed-5ad275d4e028 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:22.691209 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021a1f42-9213-4474-9fed-5ad275d4e028 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:22.691683 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021a1f42-9213-4474-9fed-5ad275d4e028 None admin] PATH_INFO: `/v3/services` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:22.692591 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-021a1f42-9213-4474-9fed-5ad275d4e028 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:50:22.693165 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-021a1f42-9213-4474-9fed-5ad275d4e028 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:22.694904 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-021a1f42-9213-4474-9fed-5ad275d4e028 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:22.701203 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 454/923] 10.222.0.27 () {56 vars in 1126 bytes} [Fri Aug 5 19:50:22 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:50:22.706542 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22b2de3-b0d4-4daf-bd35-98f66ca0be6d None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:22.814251 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22b2de3-b0d4-4daf-bd35-98f66ca0be6d None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:22.850867 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:22.854874 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000eb140-f3f2-4b72-b1ec-c7eb6c60f00c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:22.855731 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000eb140-f3f2-4b72-b1ec-c7eb6c60f00c None admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:22.856804 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000eb140-f3f2-4b72-b1ec-c7eb6c60f00c None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:22.857366 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000eb140-f3f2-4b72-b1ec-c7eb6c60f00c None admin] PATH_INFO: `/v3/endpoints` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:22.858484 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000eb140-f3f2-4b72-b1ec-c7eb6c60f00c None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:22.860066 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000eb140-f3f2-4b72-b1ec-c7eb6c60f00c None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:22.885472 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 470/924] 10.222.0.27 () {60 vars in 1144 bytes} [Fri Aug 5 19:50:22 2022] POST /identity/v3/endpoints => generated 347 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 19:50:23.475891 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b510b6ec-0863-49bb-8c89-0185a8a16beb None None] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:23.476888 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b510b6ec-0863-49bb-8c89-0185a8a16beb None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:23.477398 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b510b6ec-0863-49bb-8c89-0185a8a16beb None None] PATH_INFO: `/` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:23.479219 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 455/925] 10.222.0.27 () {54 vars in 927 bytes} [Fri Aug 5 19:50:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:50:23.484777 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a339a709-a369-4138-838c-4103152225fb None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:23.485568 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a339a709-a369-4138-838c-4103152225fb None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:23.486044 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a339a709-a369-4138-838c-4103152225fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:23.505968 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-a339a709-a369-4138-838c-4103152225fb None None] MFA Rules not processed for user `03d1834a64cd4a43829827cb99a72747`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:50:23.507763 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-a339a709-a369-4138-838c-4103152225fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:50:23.509641 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9723df61-89a1-4b1f-b252-07bde76a60dd None None] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:23.509927 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9723df61-89a1-4b1f-b252-07bde76a60dd None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:23.510204 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9723df61-89a1-4b1f-b252-07bde76a60dd None None] PATH_INFO: `/` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:23.511770 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 456/926] 10.222.0.27 () {54 vars in 927 bytes} [Fri Aug 5 19:50:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 05 19:50:23.517035 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e72f20-4655-4d1c-96e8-54167c8ca8ee None None] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:23.517330 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e72f20-4655-4d1c-96e8-54167c8ca8ee None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:23.517591 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e72f20-4655-4d1c-96e8-54167c8ca8ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:23.536471 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.auth.core [None req-72e72f20-4655-4d1c-96e8-54167c8ca8ee None None] MFA Rules not processed for user `03d1834a64cd4a43829827cb99a72747`. Rule list: `[]` (Enabled: `True`). {{(pid=80364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:50:23.538311 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-72e72f20-4655-4d1c-96e8-54167c8ca8ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:50:23.560208 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:23.598229 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 471/927] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Aug 5 19:50:23 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 05 19:50:23.618550 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a339a709-a369-4138-838c-4103152225fb None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:23.634335 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 457/928] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Aug 5 19:50:23 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 05 19:50:23.659143 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e72f20-4655-4d1c-96e8-54167c8ca8ee None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:23.737022 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a339a709-a369-4138-838c-4103152225fb None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:23.751416 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:23.772418 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e72f20-4655-4d1c-96e8-54167c8ca8ee None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:23.774724 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34e9d18-a958-43c2-a7b5-83c6da460b7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:23.775408 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34e9d18-a958-43c2-a7b5-83c6da460b7d None admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:23.775699 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34e9d18-a958-43c2-a7b5-83c6da460b7d None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:23.775972 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34e9d18-a958-43c2-a7b5-83c6da460b7d None admin] PATH_INFO: `/v3/projects` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:23.777076 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e34e9d18-a958-43c2-a7b5-83c6da460b7d None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:23.778161 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e34e9d18-a958-43c2-a7b5-83c6da460b7d None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:23.780867 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: WARNING keystone.server.flask.common [None req-e34e9d18-a958-43c2-a7b5-83c6da460b7d None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Aug 05 19:50:23.800072 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.sql.core [None req-e34e9d18-a958-43c2-a7b5-83c6da460b7d None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 05 19:50:23.800072 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Aug 05 19:50:23.800072 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [parameters: {'id': '0b25f8590c944c63ad782e5f32c1b9cc', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Aug 05 19:50:23.800072 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80363) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 05 19:50:23.815136 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0af667-ff15-4647-a1e3-bf2caadcab19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:23.816251 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0af667-ff15-4647-a1e3-bf2caadcab19 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:23.816830 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0af667-ff15-4647-a1e3-bf2caadcab19 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:23.817323 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0af667-ff15-4647-a1e3-bf2caadcab19 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:23.818279 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f0af667-ff15-4647-a1e3-bf2caadcab19 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:23.819973 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f0af667-ff15-4647-a1e3-bf2caadcab19 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:23.821122 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-e34e9d18-a958-43c2-a7b5-83c6da460b7d None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 0b25f8590c944c63ad782e5f32c1b9cc.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 0b25f8590c944c63ad782e5f32c1b9cc. Aug 05 19:50:23.821122 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:50:23.821122 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 05 19:50:23.821122 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 05 19:50:23.821122 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 05 19:50:23.821122 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 05 19:50:23.821122 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 05 19:50:23.821122 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application result = self._query(query) Aug 05 19:50:23.821122 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 05 19:50:23.821122 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application conn.query(q) Aug 05 19:50:23.821122 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 05 19:50:23.821122 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 05 19:50:23.821122 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 05 19:50:23.821915 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application result.read() Aug 05 19:50:23.821915 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 05 19:50:23.821915 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 05 19:50:23.821915 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 05 19:50:23.821915 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 05 19:50:23.821915 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 05 19:50:23.821915 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 05 19:50:23.821915 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 05 19:50:23.821915 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 05 19:50:23.821915 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 05 19:50:23.821915 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Aug 05 19:50:23.821915 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 05 19:50:23.821915 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Aug 05 19:50:23.821915 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:50:23.821915 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 05 19:50:23.821915 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 05 19:50:23.821915 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Aug 05 19:50:23.821915 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return project_ref.to_dict() Aug 05 19:50:23.822850 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 05 19:50:23.822850 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application next(self.gen) Aug 05 19:50:23.822850 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 05 19:50:23.822850 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application yield resource Aug 05 19:50:23.822850 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 05 19:50:23.822850 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application next(self.gen) Aug 05 19:50:23.822850 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 05 19:50:23.822850 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application self.session.rollback() Aug 05 19:50:23.822850 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 05 19:50:23.822850 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application self.force_reraise() Aug 05 19:50:23.822850 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 05 19:50:23.822850 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise self.value Aug 05 19:50:23.822850 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 05 19:50:23.822850 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 05 19:50:23.822850 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 05 19:50:23.822850 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application session.commit() Aug 05 19:50:23.822850 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 05 19:50:23.823767 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 05 19:50:23.823767 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 05 19:50:23.823767 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application self._prepare_impl() Aug 05 19:50:23.823767 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 05 19:50:23.823767 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application self.session.flush() Aug 05 19:50:23.823767 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 05 19:50:23.823767 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application self._flush(objects) Aug 05 19:50:23.823767 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 05 19:50:23.823767 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 05 19:50:23.823767 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 05 19:50:23.823767 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application compat.raise_( Aug 05 19:50:23.823767 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 05 19:50:23.823767 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception Aug 05 19:50:23.823767 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 05 19:50:23.823767 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application flush_context.execute() Aug 05 19:50:23.823767 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 05 19:50:23.823767 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rec.execute(self) Aug 05 19:50:23.825362 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 05 19:50:23.825362 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 05 19:50:23.825362 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 05 19:50:23.825362 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 05 19:50:23.825362 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Aug 05 19:50:23.825362 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application c = connection._execute_20( Aug 05 19:50:23.825362 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Aug 05 19:50:23.825362 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 05 19:50:23.825362 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Aug 05 19:50:23.825362 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 05 19:50:23.825362 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Aug 05 19:50:23.825362 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 05 19:50:23.825362 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Aug 05 19:50:23.825362 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 05 19:50:23.825362 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Aug 05 19:50:23.826135 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 05 19:50:23.826135 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 05 19:50:23.826135 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception Aug 05 19:50:23.826135 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 05 19:50:23.826135 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 05 19:50:23.826135 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 05 19:50:23.826135 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 05 19:50:23.826135 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 05 19:50:23.826135 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application result = self._query(query) Aug 05 19:50:23.826135 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 05 19:50:23.826135 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application conn.query(q) Aug 05 19:50:23.826135 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 05 19:50:23.826135 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 05 19:50:23.826135 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 05 19:50:23.826135 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application result.read() Aug 05 19:50:23.826135 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 05 19:50:23.826135 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 05 19:50:23.827650 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 05 19:50:23.827650 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 05 19:50:23.827650 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 05 19:50:23.827650 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 05 19:50:23.827650 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 05 19:50:23.827650 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 05 19:50:23.827650 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 05 19:50:23.827650 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Aug 05 19:50:23.827650 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [parameters: {'id': '0b25f8590c944c63ad782e5f32c1b9cc', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Aug 05 19:50:23.827650 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 05 19:50:23.827650 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Aug 05 19:50:23.827650 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 05 19:50:23.827650 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Aug 05 19:50:23.827650 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:50:23.827650 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Aug 05 19:50:23.829687 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Aug 05 19:50:23.829687 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 05 19:50:23.829687 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 05 19:50:23.829687 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Aug 05 19:50:23.829687 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Aug 05 19:50:23.829687 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 05 19:50:23.829687 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Aug 05 19:50:23.829687 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:50:23.829687 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:50:23.829687 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:50:23.829687 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:50:23.829687 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:50:23.829687 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:50:23.829687 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:50:23.829687 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:50:23.829687 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:50:23.829687 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:50:23.833212 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:50:23.833212 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Aug 05 19:50:23.833212 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Aug 05 19:50:23.833212 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:50:23.833212 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:50:23.833212 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Aug 05 19:50:23.833212 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.Conflict( Aug 05 19:50:23.833212 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 0b25f8590c944c63ad782e5f32c1b9cc. Aug 05 19:50:23.833212 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Aug 05 19:50:23.833212 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 472/929] 10.222.0.27 () {60 vars in 1140 bytes} [Fri Aug 5 19:50:23 2022] POST /identity/v3/projects => generated 262 bytes in 211 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:50:23.833212 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34e9d18-a958-43c2-a7b5-83c6da460b7d None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:23.844289 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application [None req-9f0af667-ff15-4647-a1e3-bf2caadcab19 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 05 19:50:23.844289 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:50:23.844289 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 05 19:50:23.844289 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 05 19:50:23.844289 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 05 19:50:23.844289 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 05 19:50:23.844289 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 05 19:50:23.844289 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 05 19:50:23.844289 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 05 19:50:23.844289 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Aug 05 19:50:23.844289 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 05 19:50:23.844289 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Aug 05 19:50:23.844289 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:50:23.844289 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:50:23.844289 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:50:23.844289 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:50:23.844289 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:50:23.845346 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:50:23.845346 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:50:23.845346 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:50:23.845346 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:50:23.845346 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:50:23.845346 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:50:23.845346 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 05 19:50:23.845346 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 05 19:50:23.845346 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 05 19:50:23.845346 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 05 19:50:23.845346 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:50:23.845346 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:50:23.845346 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:50:23.845346 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:50:23.845346 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:50:23.845346 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:50:23.846263 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:50:23.846263 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application with Lock( Aug 05 19:50:23.846263 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:50:23.846263 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:50:23.846263 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:50:23.846263 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:50:23.846263 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:50:23.846263 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:50:23.846263 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:50:23.846263 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:50:23.846263 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 05 19:50:23.846263 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 05 19:50:23.846263 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 05 19:50:23.846263 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Aug 05 19:50:23.849412 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 458/930] 10.222.0.27 () {56 vars in 1126 bytes} [Fri Aug 5 19:50:23 2022] GET /identity/v3/services/object-store => generated 93 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 05 19:50:23.856607 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0af667-ff15-4647-a1e3-bf2caadcab19 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:23.883741 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:23.967178 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34e9d18-a958-43c2-a7b5-83c6da460b7d None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:23.997476 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0af667-ff15-4647-a1e3-bf2caadcab19 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:24.018610 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea743d7c-addf-4748-a01b-ca73103d7f2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:24.019853 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea743d7c-addf-4748-a01b-ca73103d7f2f None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:24.020350 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea743d7c-addf-4748-a01b-ca73103d7f2f None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:24.021117 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea743d7c-addf-4748-a01b-ca73103d7f2f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:24.031771 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea743d7c-addf-4748-a01b-ca73103d7f2f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:24.034132 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea743d7c-addf-4748-a01b-ca73103d7f2f None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:24.040129 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5151f6e-7b53-409f-bb74-d90629968d84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:24.040968 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5151f6e-7b53-409f-bb74-d90629968d84 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:24.041343 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5151f6e-7b53-409f-bb74-d90629968d84 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:24.041711 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5151f6e-7b53-409f-bb74-d90629968d84 None admin] PATH_INFO: `/v3/services` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:24.042531 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5151f6e-7b53-409f-bb74-d90629968d84 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:50:24.042913 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5151f6e-7b53-409f-bb74-d90629968d84 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:24.044972 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5151f6e-7b53-409f-bb74-d90629968d84 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:24.049994 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-ea743d7c-addf-4748-a01b-ca73103d7f2f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 19:50:24.049994 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:50:24.049994 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:50:24.049994 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:50:24.049994 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:50:24.049994 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:50:24.049994 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:50:24.049994 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:50:24.049994 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:50:24.049994 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:50:24.049994 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:50:24.049994 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:50:24.049994 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 05 19:50:24.049994 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 05 19:50:24.049994 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 05 19:50:24.050883 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 05 19:50:24.050883 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:50:24.050883 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:50:24.050883 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:50:24.050883 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:50:24.050883 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:50:24.050883 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:50:24.050883 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:50:24.050883 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Aug 05 19:50:24.050883 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:50:24.050883 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:50:24.050883 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:50:24.050883 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:50:24.050883 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:50:24.050883 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:50:24.050883 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:50:24.050883 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:50:24.051785 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 05 19:50:24.051785 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 05 19:50:24.051785 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 05 19:50:24.051785 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 05 19:50:24.051785 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 05 19:50:24.051785 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 05 19:50:24.051785 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 19:50:24.051785 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Aug 05 19:50:24.053851 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 473/931] 10.222.0.27 () {56 vars in 1111 bytes} [Fri Aug 5 19:50:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:50:24.056665 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 459/932] 10.222.0.27 () {56 vars in 1122 bytes} [Fri Aug 5 19:50:23 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:50:24.059903 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea743d7c-addf-4748-a01b-ca73103d7f2f None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:24.061001 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5151f6e-7b53-409f-bb74-d90629968d84 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:24.103522 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:24.185557 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:24.214678 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5151f6e-7b53-409f-bb74-d90629968d84 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:24.228395 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea743d7c-addf-4748-a01b-ca73103d7f2f None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:24.263336 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3e6a8e-22f6-44e5-a058-5e8a895fa8ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:24.263973 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3e6a8e-22f6-44e5-a058-5e8a895fa8ca None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:24.264256 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3e6a8e-22f6-44e5-a058-5e8a895fa8ca None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:24.264554 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3e6a8e-22f6-44e5-a058-5e8a895fa8ca None admin] PATH_INFO: `/v3/services` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:24.265575 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa3e6a8e-22f6-44e5-a058-5e8a895fa8ca None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:50:24.265883 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa3e6a8e-22f6-44e5-a058-5e8a895fa8ca None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:24.267439 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa3e6a8e-22f6-44e5-a058-5e8a895fa8ca None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:24.281486 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc1af05-acfb-4c4f-9e1a-e1b262244f39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:24.281895 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 460/933] 10.222.0.27 () {56 vars in 1122 bytes} [Fri Aug 5 19:50:24 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:50:24.282740 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc1af05-acfb-4c4f-9e1a-e1b262244f39 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:24.282925 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3e6a8e-22f6-44e5-a058-5e8a895fa8ca None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:24.283486 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc1af05-acfb-4c4f-9e1a-e1b262244f39 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:24.283974 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc1af05-acfb-4c4f-9e1a-e1b262244f39 None admin] PATH_INFO: `/v3/projects` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:24.284961 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc1af05-acfb-4c4f-9e1a-e1b262244f39 None admin] RBAC: Adding query filter params (name=service) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:50:24.285496 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc1af05-acfb-4c4f-9e1a-e1b262244f39 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:24.288856 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc1af05-acfb-4c4f-9e1a-e1b262244f39 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:24.328952 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 474/934] 10.222.0.27 () {56 vars in 1112 bytes} [Fri Aug 5 19:50:24 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 19:50:24.343787 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1b06bc-c91a-495c-9ec3-909bce78500d None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:24.344285 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1b06bc-c91a-495c-9ec3-909bce78500d None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:24.344800 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1b06bc-c91a-495c-9ec3-909bce78500d None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:24.353970 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 475/935] 10.222.0.27 () {54 vars in 927 bytes} [Fri Aug 5 19:50:24 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 05 19:50:24.355343 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c59672d-788c-48f5-9518-76616483281c None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:24.355661 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c59672d-788c-48f5-9518-76616483281c None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:24.355896 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c59672d-788c-48f5-9518-76616483281c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:24.400994 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-4c59672d-788c-48f5-9518-76616483281c None None] MFA Rules not processed for user `03d1834a64cd4a43829827cb99a72747`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:50:24.404219 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-4c59672d-788c-48f5-9518-76616483281c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:50:24.429826 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3e6a8e-22f6-44e5-a058-5e8a895fa8ca None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:24.477443 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97005d66-883b-4c54-92ce-0e1840fad96d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:24.478080 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97005d66-883b-4c54-92ce-0e1840fad96d None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:24.478619 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97005d66-883b-4c54-92ce-0e1840fad96d None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:24.478998 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97005d66-883b-4c54-92ce-0e1840fad96d None admin] PATH_INFO: `/v3/endpoints` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:24.479889 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97005d66-883b-4c54-92ce-0e1840fad96d None admin] RBAC: Adding query filter params (interface=public, service_id=8b4ed76680e84a3cbd78a0a9877d5608, region_id=RegionOne) {{(pid=80364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:50:24.480309 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97005d66-883b-4c54-92ce-0e1840fad96d None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:24.482443 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97005d66-883b-4c54-92ce-0e1840fad96d None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:24.489305 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 461/936] 10.222.0.27 () {56 vars in 1251 bytes} [Fri Aug 5 19:50:24 2022] GET /identity/v3/endpoints?service_id=8b4ed76680e84a3cbd78a0a9877d5608&interface=public®ion_id=RegionOne => generated 194 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:50:24.494749 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97005d66-883b-4c54-92ce-0e1840fad96d None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:24.501507 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 476/937] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Aug 5 19:50:24 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 05 19:50:24.516563 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:24.524060 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c59672d-788c-48f5-9518-76616483281c None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:24.545846 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:24.641857 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97005d66-883b-4c54-92ce-0e1840fad96d None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:24.667325 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c59672d-788c-48f5-9518-76616483281c None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:24.678200 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea257b67-1d45-4cc1-a112-ae0b49d86053 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:24.678923 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea257b67-1d45-4cc1-a112-ae0b49d86053 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:24.679205 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea257b67-1d45-4cc1-a112-ae0b49d86053 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:24.679477 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea257b67-1d45-4cc1-a112-ae0b49d86053 None admin] PATH_INFO: `/v3/services` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:24.680131 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea257b67-1d45-4cc1-a112-ae0b49d86053 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:24.681641 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea257b67-1d45-4cc1-a112-ae0b49d86053 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:24.689089 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 462/938] 10.222.0.27 () {56 vars in 1087 bytes} [Fri Aug 5 19:50:24 2022] GET /identity/v3/services => generated 1497 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 05 19:50:24.712472 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a9ac54-273b-48f3-b652-4a34c39d6c1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:24.713288 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a9ac54-273b-48f3-b652-4a34c39d6c1f None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:24.713678 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a9ac54-273b-48f3-b652-4a34c39d6c1f None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:24.714021 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a9ac54-273b-48f3-b652-4a34c39d6c1f None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:24.731909 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a9ac54-273b-48f3-b652-4a34c39d6c1f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 19:50:24.740711 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-30a9ac54-273b-48f3-b652-4a34c39d6c1f None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 05 19:50:24.740711 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:50:24.740711 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:50:24.740711 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:50:24.740711 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:50:24.740711 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:50:24.740711 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:50:24.740711 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:50:24.740711 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:50:24.740711 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:50:24.740711 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:50:24.740711 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:50:24.740711 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 19:50:24.740711 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 19:50:24.740711 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 19:50:24.740711 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 19:50:24.742231 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 19:50:24.742231 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 19:50:24.742231 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 19:50:24.742231 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 19:50:24.742231 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:50:24.742231 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:50:24.742231 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:50:24.742231 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:50:24.742231 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:50:24.742231 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:50:24.742231 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:50:24.742231 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Aug 05 19:50:24.742231 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:50:24.742231 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:50:24.742231 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:50:24.742231 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:50:24.743844 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:50:24.743844 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:50:24.743844 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:50:24.743844 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:50:24.743844 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 19:50:24.743844 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 19:50:24.743844 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 19:50:24.743844 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 19:50:24.743844 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 19:50:24.743844 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 19:50:24.743844 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 05 19:50:24.743844 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Aug 05 19:50:24.748178 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 477/939] 10.222.0.27 () {56 vars in 1120 bytes} [Fri Aug 5 19:50:24 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 05 19:50:24.752996 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea257b67-1d45-4cc1-a112-ae0b49d86053 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:24.858499 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:24.892530 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea257b67-1d45-4cc1-a112-ae0b49d86053 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:24.930039 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0822b7d-e51a-4146-99de-fb5c2f6f8a7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:24.930921 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0822b7d-e51a-4146-99de-fb5c2f6f8a7d None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:24.931371 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0822b7d-e51a-4146-99de-fb5c2f6f8a7d None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:24.931790 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0822b7d-e51a-4146-99de-fb5c2f6f8a7d None admin] PATH_INFO: `/v3/roles` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:24.932696 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0822b7d-e51a-4146-99de-fb5c2f6f8a7d None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:50:24.933145 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0822b7d-e51a-4146-99de-fb5c2f6f8a7d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:24.935197 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0822b7d-e51a-4146-99de-fb5c2f6f8a7d None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:24.944551 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 463/940] 10.222.0.27 () {56 vars in 1115 bytes} [Fri Aug 5 19:50:24 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:50:24.952540 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a9ac54-273b-48f3-b652-4a34c39d6c1f None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:25.044680 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:25.057164 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73528759-bdd4-4a85-8d82-ca2219661a54 None None] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:25.057444 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73528759-bdd4-4a85-8d82-ca2219661a54 None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:25.057726 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73528759-bdd4-4a85-8d82-ca2219661a54 None None] PATH_INFO: `/` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:25.059958 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 464/941] 10.222.0.27 () {54 vars in 927 bytes} [Fri Aug 5 19:50:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 05 19:50:25.064970 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc74e83-2d96-48bb-99bd-464dd9809f05 None None] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:25.065257 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc74e83-2d96-48bb-99bd-464dd9809f05 None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:25.065539 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc74e83-2d96-48bb-99bd-464dd9809f05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:25.077265 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a9ac54-273b-48f3-b652-4a34c39d6c1f None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:25.084194 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.auth.core [None req-bfc74e83-2d96-48bb-99bd-464dd9809f05 None None] MFA Rules not processed for user `03d1834a64cd4a43829827cb99a72747`. Rule list: `[]` (Enabled: `True`). {{(pid=80364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:50:25.085422 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-bfc74e83-2d96-48bb-99bd-464dd9809f05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:50:25.119055 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c75bff-875b-4bb3-8df7-562cfaf5953c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:25.119852 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c75bff-875b-4bb3-8df7-562cfaf5953c None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:25.120464 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c75bff-875b-4bb3-8df7-562cfaf5953c None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:25.120788 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c75bff-875b-4bb3-8df7-562cfaf5953c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:25.134100 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c75bff-875b-4bb3-8df7-562cfaf5953c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:25.135623 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c75bff-875b-4bb3-8df7-562cfaf5953c None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:25.155689 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-48c75bff-875b-4bb3-8df7-562cfaf5953c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:50:25.155689 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:50:25.155689 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:50:25.155689 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:50:25.155689 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:50:25.155689 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:50:25.155689 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:50:25.155689 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:50:25.155689 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:50:25.155689 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:50:25.155689 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:50:25.155689 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:50:25.155689 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 19:50:25.155689 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 19:50:25.155689 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 19:50:25.156761 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 19:50:25.156761 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:50:25.156761 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:50:25.156761 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:50:25.156761 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:50:25.156761 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:50:25.156761 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:50:25.156761 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:50:25.156761 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Aug 05 19:50:25.156761 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:50:25.156761 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:50:25.156761 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:50:25.156761 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:50:25.156761 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:50:25.156761 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:50:25.156761 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:50:25.156761 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:50:25.157633 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 19:50:25.157633 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 19:50:25.157633 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:50:25.157633 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Aug 05 19:50:25.157864 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 478/942] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Aug 5 19:50:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:50:25.162951 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c75bff-875b-4bb3-8df7-562cfaf5953c None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:25.170401 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 465/943] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Aug 5 19:50:25 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 05 19:50:25.193254 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc74e83-2d96-48bb-99bd-464dd9809f05 None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:25.275292 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:25.300857 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c75bff-875b-4bb3-8df7-562cfaf5953c None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:25.320126 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc74e83-2d96-48bb-99bd-464dd9809f05 None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:25.334895 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:25.344184 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5b3adc-8a87-4565-a914-98f0c430f34e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:25.344950 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5b3adc-8a87-4565-a914-98f0c430f34e None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:25.345294 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5b3adc-8a87-4565-a914-98f0c430f34e None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:25.345671 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5b3adc-8a87-4565-a914-98f0c430f34e None admin] PATH_INFO: `/v3/domains` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:25.346463 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a5b3adc-8a87-4565-a914-98f0c430f34e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:50:25.346842 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a5b3adc-8a87-4565-a914-98f0c430f34e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:25.348310 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a5b3adc-8a87-4565-a914-98f0c430f34e None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:25.374955 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2401588d-1aed-40ff-8ffd-fa1d517b72a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:25.375478 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 479/944] 10.222.0.27 () {56 vars in 1109 bytes} [Fri Aug 5 19:50:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 19:50:25.376077 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2401588d-1aed-40ff-8ffd-fa1d517b72a7 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:25.376487 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2401588d-1aed-40ff-8ffd-fa1d517b72a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:25.376868 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2401588d-1aed-40ff-8ffd-fa1d517b72a7 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:25.377610 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2401588d-1aed-40ff-8ffd-fa1d517b72a7 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:25.379134 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2401588d-1aed-40ff-8ffd-fa1d517b72a7 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:25.380884 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5b3adc-8a87-4565-a914-98f0c430f34e None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:25.391446 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application [None req-2401588d-1aed-40ff-8ffd-fa1d517b72a7 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 05 19:50:25.391446 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:50:25.391446 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 05 19:50:25.391446 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 05 19:50:25.391446 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 05 19:50:25.391446 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 05 19:50:25.391446 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 05 19:50:25.391446 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 05 19:50:25.391446 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 05 19:50:25.391446 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Aug 05 19:50:25.391446 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 05 19:50:25.391446 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Aug 05 19:50:25.391446 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:50:25.391446 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:50:25.391446 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:50:25.391446 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:50:25.391446 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:50:25.392522 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:50:25.392522 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:50:25.392522 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:50:25.392522 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:50:25.392522 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:50:25.392522 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:50:25.392522 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 05 19:50:25.392522 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 05 19:50:25.392522 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 05 19:50:25.392522 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 05 19:50:25.392522 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:50:25.392522 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:50:25.392522 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:50:25.392522 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:50:25.392522 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:50:25.392522 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:50:25.393408 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:50:25.393408 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application with Lock( Aug 05 19:50:25.393408 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:50:25.393408 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:50:25.393408 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:50:25.393408 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:50:25.393408 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:50:25.393408 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:50:25.393408 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:50:25.393408 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:50:25.393408 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 05 19:50:25.393408 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 05 19:50:25.393408 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 05 19:50:25.393408 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Aug 05 19:50:25.396579 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 466/945] 10.222.0.27 () {56 vars in 1111 bytes} [Fri Aug 5 19:50:25 2022] GET /identity/v3/services/compute => generated 88 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 05 19:50:25.400652 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2401588d-1aed-40ff-8ffd-fa1d517b72a7 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:25.538228 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2401588d-1aed-40ff-8ffd-fa1d517b72a7 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:25.541790 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5b3adc-8a87-4565-a914-98f0c430f34e None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:25.583806 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:25.595445 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a995f516-8673-42d2-a0ad-52fc4acaaa64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:25.596601 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a995f516-8673-42d2-a0ad-52fc4acaaa64 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:25.596949 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a995f516-8673-42d2-a0ad-52fc4acaaa64 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:25.597316 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a995f516-8673-42d2-a0ad-52fc4acaaa64 None admin] PATH_INFO: `/v3/services` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:25.598294 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a995f516-8673-42d2-a0ad-52fc4acaaa64 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:50:25.598669 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a995f516-8673-42d2-a0ad-52fc4acaaa64 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:25.600097 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a995f516-8673-42d2-a0ad-52fc4acaaa64 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:25.603035 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b209c3-5ade-433d-be8d-f7395391d841 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:25.604244 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b209c3-5ade-433d-be8d-f7395391d841 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:25.604748 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b209c3-5ade-433d-be8d-f7395391d841 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:25.605202 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b209c3-5ade-433d-be8d-f7395391d841 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:25.608653 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 467/946] 10.222.0.27 () {56 vars in 1112 bytes} [Fri Aug 5 19:50:25 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 19:50:25.612213 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a995f516-8673-42d2-a0ad-52fc4acaaa64 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:25.617593 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b209c3-5ade-433d-be8d-f7395391d841 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:25.619297 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b209c3-5ade-433d-be8d-f7395391d841 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:25.632787 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-a3b209c3-5ade-433d-be8d-f7395391d841 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 05 19:50:25.632787 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:50:25.632787 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:50:25.632787 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:50:25.632787 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:50:25.632787 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:50:25.632787 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:50:25.632787 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:50:25.632787 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:50:25.632787 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:50:25.632787 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:50:25.632787 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:50:25.632787 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 05 19:50:25.632787 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 05 19:50:25.632787 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 05 19:50:25.632787 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 05 19:50:25.633788 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:50:25.633788 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:50:25.633788 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 05 19:50:25.633788 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 19:50:25.633788 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 05 19:50:25.633788 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 19:50:25.633788 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:50:25.633788 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:50:25.633788 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:50:25.633788 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:50:25.633788 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:50:25.633788 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Aug 05 19:50:25.633788 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:50:25.633788 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:50:25.633788 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:50:25.633788 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:50:25.633788 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:50:25.633788 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:50:25.634716 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:50:25.634716 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:50:25.634716 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 05 19:50:25.634716 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 05 19:50:25.634716 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 05 19:50:25.634716 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 05 19:50:25.634716 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 05 19:50:25.634716 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 05 19:50:25.634716 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 05 19:50:25.634716 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Aug 05 19:50:25.638045 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 480/947] 10.222.0.27 () {56 vars in 1117 bytes} [Fri Aug 5 19:50:25 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:50:25.643698 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b209c3-5ade-433d-be8d-f7395391d841 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:25.705957 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:25.771129 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a995f516-8673-42d2-a0ad-52fc4acaaa64 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:25.778044 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b209c3-5ade-433d-be8d-f7395391d841 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:25.820752 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22bf189-a093-42df-ae52-1b5cc9c888c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:25.821374 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22bf189-a093-42df-ae52-1b5cc9c888c4 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:25.821650 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22bf189-a093-42df-ae52-1b5cc9c888c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:25.821919 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22bf189-a093-42df-ae52-1b5cc9c888c4 None admin] PATH_INFO: `/v3/users` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:25.822719 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22bf189-a093-42df-ae52-1b5cc9c888c4 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:50:25.823021 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22bf189-a093-42df-ae52-1b5cc9c888c4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:25.824639 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22bf189-a093-42df-ae52-1b5cc9c888c4 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:25.827147 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0281bca8-a7ad-4f47-88e3-ee76a9918bc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:25.828417 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0281bca8-a7ad-4f47-88e3-ee76a9918bc4 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:25.829298 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0281bca8-a7ad-4f47-88e3-ee76a9918bc4 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:25.830004 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0281bca8-a7ad-4f47-88e3-ee76a9918bc4 None admin] PATH_INFO: `/v3/services` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:25.831366 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0281bca8-a7ad-4f47-88e3-ee76a9918bc4 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:50:25.832109 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0281bca8-a7ad-4f47-88e3-ee76a9918bc4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:25.834731 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0281bca8-a7ad-4f47-88e3-ee76a9918bc4 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:25.845907 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 481/949] 10.222.0.27 () {56 vars in 1149 bytes} [Fri Aug 5 19:50:25 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 19:50:25.846275 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 468/949] 10.222.0.27 () {56 vars in 1112 bytes} [Fri Aug 5 19:50:25 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:50:25.850337 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22bf189-a093-42df-ae52-1b5cc9c888c4 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:25.974765 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22bf189-a093-42df-ae52-1b5cc9c888c4 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:25.987053 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:26.017396 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a4aac0-6746-4d51-8d6c-272d4d56a8d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:26.018045 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a4aac0-6746-4d51-8d6c-272d4d56a8d3 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:26.018322 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a4aac0-6746-4d51-8d6c-272d4d56a8d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:26.018595 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a4aac0-6746-4d51-8d6c-272d4d56a8d3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:26.030325 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a4aac0-6746-4d51-8d6c-272d4d56a8d3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:26.031767 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a4aac0-6746-4d51-8d6c-272d4d56a8d3 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:26.053380 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-c8a4aac0-6746-4d51-8d6c-272d4d56a8d3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:50:26.053380 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:50:26.053380 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:50:26.053380 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:50:26.053380 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:50:26.053380 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:50:26.053380 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:50:26.053380 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:50:26.053380 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:50:26.053380 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:50:26.053380 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:50:26.053380 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:50:26.053380 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 19:50:26.053380 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 19:50:26.053380 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 19:50:26.054394 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 19:50:26.054394 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:50:26.054394 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:50:26.054394 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:50:26.054394 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:50:26.054394 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:50:26.054394 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:50:26.054394 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:50:26.054394 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Aug 05 19:50:26.054394 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:50:26.054394 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:50:26.054394 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:50:26.054394 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:50:26.054394 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:50:26.054394 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:50:26.054394 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:50:26.054394 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:50:26.055307 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 19:50:26.055307 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 19:50:26.055307 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:50:26.055307 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Aug 05 19:50:26.055307 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 482/950] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Aug 5 19:50:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:50:26.060707 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0281bca8-a7ad-4f47-88e3-ee76a9918bc4 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:26.119811 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:26.165069 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0281bca8-a7ad-4f47-88e3-ee76a9918bc4 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:26.201730 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c04167-e095-4fd0-b3f5-d31882d6485e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:26.202619 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c04167-e095-4fd0-b3f5-d31882d6485e None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:26.202900 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c04167-e095-4fd0-b3f5-d31882d6485e None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:26.203173 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c04167-e095-4fd0-b3f5-d31882d6485e None admin] PATH_INFO: `/v3/domains` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:26.203899 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c04167-e095-4fd0-b3f5-d31882d6485e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:50:26.204200 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c04167-e095-4fd0-b3f5-d31882d6485e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:26.205683 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c04167-e095-4fd0-b3f5-d31882d6485e None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:26.224122 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 469/951] 10.222.0.27 () {56 vars in 1109 bytes} [Fri Aug 5 19:50:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:50:26.225530 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a4aac0-6746-4d51-8d6c-272d4d56a8d3 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:26.329998 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a4aac0-6746-4d51-8d6c-272d4d56a8d3 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:26.370487 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a36e025-cb53-4192-81b3-fc42294170a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:26.371095 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a36e025-cb53-4192-81b3-fc42294170a1 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:26.371374 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a36e025-cb53-4192-81b3-fc42294170a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:26.372180 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a36e025-cb53-4192-81b3-fc42294170a1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:26.379425 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a36e025-cb53-4192-81b3-fc42294170a1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:26.381005 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a36e025-cb53-4192-81b3-fc42294170a1 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:26.393635 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-9a36e025-cb53-4192-81b3-fc42294170a1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 19:50:26.393635 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:50:26.393635 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:50:26.393635 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:50:26.393635 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:50:26.393635 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:50:26.393635 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:50:26.393635 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:50:26.393635 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:50:26.393635 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:50:26.393635 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:50:26.393635 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:50:26.393635 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 05 19:50:26.393635 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 05 19:50:26.393635 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 05 19:50:26.394592 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 05 19:50:26.394592 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:50:26.394592 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:50:26.394592 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:50:26.394592 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:50:26.394592 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:50:26.394592 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:50:26.394592 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:50:26.394592 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Aug 05 19:50:26.394592 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:50:26.394592 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:50:26.394592 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:50:26.394592 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:50:26.394592 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:50:26.394592 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:50:26.394592 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:50:26.394592 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:50:26.395470 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 05 19:50:26.395470 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 05 19:50:26.395470 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 05 19:50:26.395470 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 05 19:50:26.395470 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 05 19:50:26.395470 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 05 19:50:26.395470 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 19:50:26.395470 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Aug 05 19:50:26.400313 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 483/952] 10.222.0.27 () {56 vars in 1111 bytes} [Fri Aug 5 19:50:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:50:26.401745 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c04167-e095-4fd0-b3f5-d31882d6485e None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:26.509843 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c04167-e095-4fd0-b3f5-d31882d6485e None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:26.529196 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:26.546982 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679bde91-cd54-4353-b330-9c013db27bd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:26.547742 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679bde91-cd54-4353-b330-9c013db27bd6 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:26.547903 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679bde91-cd54-4353-b330-9c013db27bd6 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:26.548221 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679bde91-cd54-4353-b330-9c013db27bd6 None admin] PATH_INFO: `/v3/projects` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:26.549011 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679bde91-cd54-4353-b330-9c013db27bd6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:50:26.549312 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679bde91-cd54-4353-b330-9c013db27bd6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:26.550737 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679bde91-cd54-4353-b330-9c013db27bd6 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:26.567327 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 470/953] 10.222.0.27 () {56 vars in 1148 bytes} [Fri Aug 5 19:50:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:50:26.572967 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a36e025-cb53-4192-81b3-fc42294170a1 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:26.610653 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:26.683586 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe612061-c11f-434d-93e3-be49fbd75c80 None None] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:26.684158 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe612061-c11f-434d-93e3-be49fbd75c80 None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:26.684761 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe612061-c11f-434d-93e3-be49fbd75c80 None None] PATH_INFO: `/` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:26.687422 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 471/954] 10.222.0.27 () {54 vars in 927 bytes} [Fri Aug 5 19:50:26 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:50:26.693025 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a61430-a575-4c42-bd50-1b51e13c423b None None] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:26.693600 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a61430-a575-4c42-bd50-1b51e13c423b None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:26.694107 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a61430-a575-4c42-bd50-1b51e13c423b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:26.721914 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.auth.core [None req-45a61430-a575-4c42-bd50-1b51e13c423b None None] MFA Rules not processed for user `03d1834a64cd4a43829827cb99a72747`. Rule list: `[]` (Enabled: `True`). {{(pid=80364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:50:26.723984 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-45a61430-a575-4c42-bd50-1b51e13c423b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:50:26.725592 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a36e025-cb53-4192-81b3-fc42294170a1 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:26.787445 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14306668-5b12-47dd-ac5a-9f8644ed1a8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:26.788151 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14306668-5b12-47dd-ac5a-9f8644ed1a8c None admin] REQUEST_METHOD: `PUT` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:26.788476 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14306668-5b12-47dd-ac5a-9f8644ed1a8c None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:26.788775 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14306668-5b12-47dd-ac5a-9f8644ed1a8c None admin] PATH_INFO: `/v3/projects/a70a82dcfc574fad90b3f5f00abbba58/users/bdf7cf05a77e4298b2bf4f58b40f54aa/roles/31647643dd7c43aba270f63d5fe83acb` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:26.794442 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14306668-5b12-47dd-ac5a-9f8644ed1a8c None admin] RBAC: Authorizing `identity:create_grant(project_id=a70a82dcfc574fad90b3f5f00abbba58, user_id=bdf7cf05a77e4298b2bf4f58b40f54aa, role_id=31647643dd7c43aba270f63d5fe83acb)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:26.794442 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14306668-5b12-47dd-ac5a-9f8644ed1a8c None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:26.820620 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 484/955] 10.222.0.27 () {58 vars in 1439 bytes} [Fri Aug 5 19:50:26 2022] PUT /identity/v3/projects/a70a82dcfc574fad90b3f5f00abbba58/users/bdf7cf05a77e4298b2bf4f58b40f54aa/roles/31647643dd7c43aba270f63d5fe83acb => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 19:50:26.827952 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 472/956] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Aug 5 19:50:26 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 05 19:50:26.854079 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14306668-5b12-47dd-ac5a-9f8644ed1a8c None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:26.880542 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7c0058-1aef-461f-b5bb-e93b8223e77e None None] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:26.881215 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7c0058-1aef-461f-b5bb-e93b8223e77e None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:26.881777 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7c0058-1aef-461f-b5bb-e93b8223e77e None None] PATH_INFO: `/` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:26.883753 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 473/957] 10.222.0.27 () {54 vars in 927 bytes} [Fri Aug 5 19:50:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 05 19:50:26.890339 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007a12ce-050d-41b3-b25a-79d61cce1528 None None] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:26.890650 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007a12ce-050d-41b3-b25a-79d61cce1528 None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:26.890909 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007a12ce-050d-41b3-b25a-79d61cce1528 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:26.913922 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.auth.core [None req-007a12ce-050d-41b3-b25a-79d61cce1528 None None] MFA Rules not processed for user `03d1834a64cd4a43829827cb99a72747`. Rule list: `[]` (Enabled: `True`). {{(pid=80364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:50:26.915208 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-007a12ce-050d-41b3-b25a-79d61cce1528 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:50:26.972893 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:27.001083 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14306668-5b12-47dd-ac5a-9f8644ed1a8c None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:27.011184 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 474/958] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Aug 5 19:50:26 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 05 19:50:27.040323 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007a12ce-050d-41b3-b25a-79d61cce1528 None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:27.046454 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cdb6c7c-f017-4d7d-905c-0453e4dad9f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:27.047180 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdb6c7c-f017-4d7d-905c-0453e4dad9f5 None admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:27.047524 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdb6c7c-f017-4d7d-905c-0453e4dad9f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:27.047867 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdb6c7c-f017-4d7d-905c-0453e4dad9f5 None admin] PATH_INFO: `/v3/users` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:27.048720 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cdb6c7c-f017-4d7d-905c-0453e4dad9f5 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:27.050313 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cdb6c7c-f017-4d7d-905c-0453e4dad9f5 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:27.066887 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.sql.core [None req-7cdb6c7c-f017-4d7d-905c-0453e4dad9f5 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 05 19:50:27.066887 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Aug 05 19:50:27.066887 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [parameters: {'user_id': '516bf66f71834b108d9e83c50ee89b53', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Aug 05 19:50:27.066887 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80363) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 05 19:50:27.078307 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-7cdb6c7c-f017-4d7d-905c-0453e4dad9f5 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Aug 05 19:50:27.078307 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:50:27.078307 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 05 19:50:27.078307 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 05 19:50:27.078307 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 05 19:50:27.078307 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 05 19:50:27.078307 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 05 19:50:27.078307 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application result = self._query(query) Aug 05 19:50:27.078307 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 05 19:50:27.078307 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application conn.query(q) Aug 05 19:50:27.078307 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 05 19:50:27.078307 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 05 19:50:27.078307 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 05 19:50:27.078307 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application result.read() Aug 05 19:50:27.078307 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 05 19:50:27.079316 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 05 19:50:27.079316 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 05 19:50:27.079316 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 05 19:50:27.079316 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 05 19:50:27.079316 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 05 19:50:27.079316 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 05 19:50:27.079316 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 05 19:50:27.079316 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 05 19:50:27.079316 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Aug 05 19:50:27.079316 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 05 19:50:27.079316 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Aug 05 19:50:27.079316 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:50:27.079316 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 05 19:50:27.079316 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 05 19:50:27.079316 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Aug 05 19:50:27.079316 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Aug 05 19:50:27.079316 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 05 19:50:27.079316 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application next(self.gen) Aug 05 19:50:27.080335 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 05 19:50:27.080335 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application yield resource Aug 05 19:50:27.080335 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 05 19:50:27.080335 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application next(self.gen) Aug 05 19:50:27.080335 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 05 19:50:27.080335 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application self.session.rollback() Aug 05 19:50:27.080335 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 05 19:50:27.080335 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application self.force_reraise() Aug 05 19:50:27.080335 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 05 19:50:27.080335 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise self.value Aug 05 19:50:27.080335 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 05 19:50:27.080335 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 05 19:50:27.080335 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 05 19:50:27.080335 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application session.commit() Aug 05 19:50:27.080335 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 05 19:50:27.080335 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 05 19:50:27.080335 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 05 19:50:27.086475 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application self._prepare_impl() Aug 05 19:50:27.086475 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 05 19:50:27.086475 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application self.session.flush() Aug 05 19:50:27.086475 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 05 19:50:27.086475 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application self._flush(objects) Aug 05 19:50:27.086475 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 05 19:50:27.086475 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 05 19:50:27.086475 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 05 19:50:27.086475 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application compat.raise_( Aug 05 19:50:27.086475 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 05 19:50:27.086475 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception Aug 05 19:50:27.086475 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 05 19:50:27.086475 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application flush_context.execute() Aug 05 19:50:27.086475 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 05 19:50:27.086475 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rec.execute(self) Aug 05 19:50:27.086475 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 05 19:50:27.086475 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 05 19:50:27.087392 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 05 19:50:27.087392 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 05 19:50:27.087392 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Aug 05 19:50:27.087392 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application result = connection._execute_20( Aug 05 19:50:27.087392 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Aug 05 19:50:27.087392 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 05 19:50:27.087392 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Aug 05 19:50:27.087392 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 05 19:50:27.087392 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Aug 05 19:50:27.087392 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 05 19:50:27.087392 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Aug 05 19:50:27.087392 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 05 19:50:27.087392 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Aug 05 19:50:27.087392 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 05 19:50:27.087392 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 05 19:50:27.087392 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception Aug 05 19:50:27.088311 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 05 19:50:27.088311 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 05 19:50:27.088311 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 05 19:50:27.088311 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 05 19:50:27.088311 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 05 19:50:27.088311 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application result = self._query(query) Aug 05 19:50:27.088311 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 05 19:50:27.088311 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application conn.query(q) Aug 05 19:50:27.088311 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 05 19:50:27.088311 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 05 19:50:27.088311 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 05 19:50:27.088311 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application result.read() Aug 05 19:50:27.088311 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 05 19:50:27.088311 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 05 19:50:27.088311 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 05 19:50:27.088311 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 05 19:50:27.090846 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 05 19:50:27.090846 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 05 19:50:27.090846 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 05 19:50:27.090846 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 05 19:50:27.090846 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 05 19:50:27.090846 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Aug 05 19:50:27.090846 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [parameters: {'user_id': '516bf66f71834b108d9e83c50ee89b53', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Aug 05 19:50:27.090846 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 05 19:50:27.090846 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Aug 05 19:50:27.090846 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 05 19:50:27.090846 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Aug 05 19:50:27.090846 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:50:27.090846 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:50:27.090846 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:50:27.090846 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:50:27.092985 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:50:27.092985 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:50:27.092985 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:50:27.092985 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:50:27.092985 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:50:27.092985 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:50:27.092985 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:50:27.092985 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Aug 05 19:50:27.092985 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Aug 05 19:50:27.092985 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:50:27.092985 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:50:27.092985 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 05 19:50:27.092985 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 19:50:27.092985 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 05 19:50:27.092985 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 19:50:27.092985 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Aug 05 19:50:27.092985 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Aug 05 19:50:27.095553 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Aug 05 19:50:27.095553 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Aug 05 19:50:27.095553 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 05 19:50:27.095553 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 05 19:50:27.095553 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Aug 05 19:50:27.095553 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Aug 05 19:50:27.095553 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 485/959] 10.222.0.27 () {60 vars in 1131 bytes} [Fri Aug 5 19:50:26 2022] POST /identity/v3/users => generated 160 bytes in 226 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:50:27.095553 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cdb6c7c-f017-4d7d-905c-0453e4dad9f5 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:27.108877 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:27.171440 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007a12ce-050d-41b3-b25a-79d61cce1528 None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:27.216775 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493dfadb-e286-46b4-8a8e-d1033741c06f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:27.217816 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493dfadb-e286-46b4-8a8e-d1033741c06f None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:27.218299 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493dfadb-e286-46b4-8a8e-d1033741c06f None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:27.218774 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493dfadb-e286-46b4-8a8e-d1033741c06f None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:27.219665 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-493dfadb-e286-46b4-8a8e-d1033741c06f None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:27.221329 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-493dfadb-e286-46b4-8a8e-d1033741c06f None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:27.227764 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cdb6c7c-f017-4d7d-905c-0453e4dad9f5 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:27.231728 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application [None req-493dfadb-e286-46b4-8a8e-d1033741c06f None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 05 19:50:27.231728 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:50:27.231728 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 05 19:50:27.231728 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 05 19:50:27.231728 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 05 19:50:27.231728 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 05 19:50:27.231728 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 05 19:50:27.231728 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 05 19:50:27.231728 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 05 19:50:27.231728 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Aug 05 19:50:27.231728 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 05 19:50:27.231728 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Aug 05 19:50:27.231728 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:50:27.231728 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:50:27.231728 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:50:27.231728 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:50:27.231728 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:50:27.232804 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:50:27.232804 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:50:27.232804 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:50:27.232804 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:50:27.232804 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:50:27.232804 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:50:27.232804 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 05 19:50:27.232804 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 05 19:50:27.232804 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 05 19:50:27.232804 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 05 19:50:27.232804 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:50:27.232804 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:50:27.232804 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:50:27.232804 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:50:27.232804 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:50:27.232804 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:50:27.233688 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:50:27.233688 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application with Lock( Aug 05 19:50:27.233688 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:50:27.233688 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:50:27.233688 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:50:27.233688 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:50:27.233688 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:50:27.233688 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:50:27.233688 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:50:27.233688 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:50:27.233688 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 05 19:50:27.233688 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 05 19:50:27.233688 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 05 19:50:27.233688 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Aug 05 19:50:27.241120 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 475/960] 10.222.0.27 () {56 vars in 1126 bytes} [Fri Aug 5 19:50:27 2022] GET /identity/v3/services/object-store => generated 93 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 05 19:50:27.242298 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493dfadb-e286-46b4-8a8e-d1033741c06f None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:27.264387 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77764cdf-4932-4bff-846d-def976454f69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:27.265058 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77764cdf-4932-4bff-846d-def976454f69 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:27.265336 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77764cdf-4932-4bff-846d-def976454f69 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:27.265606 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77764cdf-4932-4bff-846d-def976454f69 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:27.275493 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77764cdf-4932-4bff-846d-def976454f69 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:27.277180 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77764cdf-4932-4bff-846d-def976454f69 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:27.288753 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-77764cdf-4932-4bff-846d-def976454f69 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 05 19:50:27.288753 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:50:27.288753 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:50:27.288753 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:50:27.288753 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:50:27.288753 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:50:27.288753 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:50:27.288753 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:50:27.288753 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:50:27.288753 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:50:27.288753 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:50:27.288753 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:50:27.288753 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 05 19:50:27.288753 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 05 19:50:27.288753 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 05 19:50:27.288753 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 05 19:50:27.289860 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:50:27.289860 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:50:27.289860 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 05 19:50:27.289860 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 19:50:27.289860 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 05 19:50:27.289860 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 19:50:27.289860 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:50:27.289860 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:50:27.289860 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:50:27.289860 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:50:27.289860 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:50:27.289860 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Aug 05 19:50:27.289860 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:50:27.289860 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:50:27.289860 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:50:27.289860 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:50:27.289860 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:50:27.289860 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:50:27.291155 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:50:27.291155 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:50:27.291155 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 05 19:50:27.291155 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 05 19:50:27.291155 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 05 19:50:27.291155 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 05 19:50:27.291155 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 05 19:50:27.291155 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 05 19:50:27.291155 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 05 19:50:27.291155 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Aug 05 19:50:27.295267 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 486/961] 10.222.0.27 () {56 vars in 1099 bytes} [Fri Aug 5 19:50:27 2022] GET /identity/v3/users/cinder => generated 84 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:50:27.296668 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77764cdf-4932-4bff-846d-def976454f69 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:27.299755 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:27.379235 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:27.384112 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493dfadb-e286-46b4-8a8e-d1033741c06f None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:27.414530 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77764cdf-4932-4bff-846d-def976454f69 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:27.429424 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9edfd35e-7ea0-4109-b5f9-72b370b7bc6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:27.430808 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edfd35e-7ea0-4109-b5f9-72b370b7bc6f None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:27.431551 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edfd35e-7ea0-4109-b5f9-72b370b7bc6f None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:27.432028 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edfd35e-7ea0-4109-b5f9-72b370b7bc6f None admin] PATH_INFO: `/v3/services` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:27.433132 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9edfd35e-7ea0-4109-b5f9-72b370b7bc6f None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:50:27.433444 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9edfd35e-7ea0-4109-b5f9-72b370b7bc6f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:27.435448 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9edfd35e-7ea0-4109-b5f9-72b370b7bc6f None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:27.450762 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 476/962] 10.222.0.27 () {56 vars in 1122 bytes} [Fri Aug 5 19:50:27 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:50:27.451787 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9edfd35e-7ea0-4109-b5f9-72b370b7bc6f None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:27.463168 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11964581-b75e-48a4-9b46-1421796e7191 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:27.463792 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11964581-b75e-48a4-9b46-1421796e7191 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:27.464072 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11964581-b75e-48a4-9b46-1421796e7191 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:27.464723 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11964581-b75e-48a4-9b46-1421796e7191 None admin] PATH_INFO: `/v3/users` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:27.465224 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11964581-b75e-48a4-9b46-1421796e7191 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:50:27.465522 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11964581-b75e-48a4-9b46-1421796e7191 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:27.466998 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11964581-b75e-48a4-9b46-1421796e7191 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:27.520879 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 487/963] 10.222.0.27 () {56 vars in 1101 bytes} [Fri Aug 5 19:50:27 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 19:50:27.580594 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9edfd35e-7ea0-4109-b5f9-72b370b7bc6f None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:27.621231 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e301efb-7227-4b74-a3db-3c0626477bc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:27.622311 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e301efb-7227-4b74-a3db-3c0626477bc2 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:27.622794 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e301efb-7227-4b74-a3db-3c0626477bc2 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:27.623267 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e301efb-7227-4b74-a3db-3c0626477bc2 None admin] PATH_INFO: `/v3/services` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:27.624160 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e301efb-7227-4b74-a3db-3c0626477bc2 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:50:27.624699 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e301efb-7227-4b74-a3db-3c0626477bc2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:27.626317 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e301efb-7227-4b74-a3db-3c0626477bc2 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:27.633217 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 477/964] 10.222.0.27 () {56 vars in 1122 bytes} [Fri Aug 5 19:50:27 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:50:27.639954 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11964581-b75e-48a4-9b46-1421796e7191 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:27.710539 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:27.753674 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11964581-b75e-48a4-9b46-1421796e7191 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:27.792724 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d418587-1859-48a4-b4fb-4825ab02c33f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:27.793488 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d418587-1859-48a4-b4fb-4825ab02c33f None admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:27.793769 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d418587-1859-48a4-b4fb-4825ab02c33f None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:27.794044 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d418587-1859-48a4-b4fb-4825ab02c33f None admin] PATH_INFO: `/v3/endpoints` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:27.794829 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d418587-1859-48a4-b4fb-4825ab02c33f None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:27.796216 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d418587-1859-48a4-b4fb-4825ab02c33f None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:27.817095 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 488/965] 10.222.0.27 () {60 vars in 1144 bytes} [Fri Aug 5 19:50:27 2022] POST /identity/v3/endpoints => generated 349 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 19:50:28.018538 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390a05da-1107-432d-9f23-b6f3702f1e24 None None] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:28.018862 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390a05da-1107-432d-9f23-b6f3702f1e24 None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:28.019101 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390a05da-1107-432d-9f23-b6f3702f1e24 None None] PATH_INFO: `/` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:28.020632 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 478/966] 10.222.0.27 () {54 vars in 927 bytes} [Fri Aug 5 19:50:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:50:28.026023 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fdd708-e4cf-4e73-8891-4207d1a5fd23 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:28.026314 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fdd708-e4cf-4e73-8891-4207d1a5fd23 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:28.026592 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fdd708-e4cf-4e73-8891-4207d1a5fd23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:28.045716 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-d5fdd708-e4cf-4e73-8891-4207d1a5fd23 None None] MFA Rules not processed for user `03d1834a64cd4a43829827cb99a72747`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:50:28.047162 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-d5fdd708-e4cf-4e73-8891-4207d1a5fd23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:50:28.121540 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 489/967] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Aug 5 19:50:28 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 05 19:50:28.143983 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390a05da-1107-432d-9f23-b6f3702f1e24 None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:28.174384 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:28.256862 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390a05da-1107-432d-9f23-b6f3702f1e24 None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:28.294225 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7098f7-c379-4e2d-b3f3-5eb76c1c3d13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:28.294917 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7098f7-c379-4e2d-b3f3-5eb76c1c3d13 None admin] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:28.295202 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7098f7-c379-4e2d-b3f3-5eb76c1c3d13 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:28.295469 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7098f7-c379-4e2d-b3f3-5eb76c1c3d13 None admin] PATH_INFO: `/v3/services` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:28.296285 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d7098f7-c379-4e2d-b3f3-5eb76c1c3d13 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:28.297646 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d7098f7-c379-4e2d-b3f3-5eb76c1c3d13 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:28.316667 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 479/968] 10.222.0.27 () {60 vars in 1141 bytes} [Fri Aug 5 19:50:28 2022] POST /identity/v3/services => generated 242 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 19:50:28.894804 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b9faf8-4ef6-4c47-8133-32abf5c27ea1 None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:28.895341 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b9faf8-4ef6-4c47-8133-32abf5c27ea1 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:28.895341 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b9faf8-4ef6-4c47-8133-32abf5c27ea1 None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:28.896812 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 490/969] 10.222.0.27 () {54 vars in 927 bytes} [Fri Aug 5 19:50:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:50:28.903465 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df140fb-e90d-4639-93ea-1f4c5174aa78 None None] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:28.903990 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df140fb-e90d-4639-93ea-1f4c5174aa78 None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:28.903990 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df140fb-e90d-4639-93ea-1f4c5174aa78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:28.923439 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.auth.core [None req-6df140fb-e90d-4639-93ea-1f4c5174aa78 None None] MFA Rules not processed for user `03d1834a64cd4a43829827cb99a72747`. Rule list: `[]` (Enabled: `True`). {{(pid=80364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:50:28.924679 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-6df140fb-e90d-4639-93ea-1f4c5174aa78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:50:29.008531 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 480/970] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Aug 5 19:50:28 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 05 19:50:29.030353 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b9faf8-4ef6-4c47-8133-32abf5c27ea1 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:29.079493 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:29.146337 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b9faf8-4ef6-4c47-8133-32abf5c27ea1 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:29.181976 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25fc364e-20a8-49af-8640-9f9f690b0276 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:29.182629 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fc364e-20a8-49af-8640-9f9f690b0276 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:29.182907 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fc364e-20a8-49af-8640-9f9f690b0276 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:29.183177 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fc364e-20a8-49af-8640-9f9f690b0276 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:29.195963 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25fc364e-20a8-49af-8640-9f9f690b0276 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 19:50:29.201437 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-25fc364e-20a8-49af-8640-9f9f690b0276 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 05 19:50:29.201437 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:50:29.201437 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:50:29.201437 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:50:29.201437 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:50:29.201437 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:50:29.201437 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:50:29.201437 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:50:29.201437 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:50:29.201437 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:50:29.201437 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:50:29.201437 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:50:29.201437 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 19:50:29.201437 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 19:50:29.201437 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 19:50:29.201437 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 19:50:29.202664 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 19:50:29.202664 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 19:50:29.202664 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 19:50:29.202664 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 19:50:29.202664 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:50:29.202664 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:50:29.202664 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:50:29.202664 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:50:29.202664 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:50:29.202664 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:50:29.202664 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:50:29.202664 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Aug 05 19:50:29.202664 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:50:29.202664 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:50:29.202664 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:50:29.202664 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:50:29.203536 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:50:29.203536 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:50:29.203536 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:50:29.203536 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:50:29.203536 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 19:50:29.203536 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 19:50:29.203536 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 19:50:29.203536 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 19:50:29.203536 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 19:50:29.203536 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 19:50:29.203536 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 05 19:50:29.203536 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Aug 05 19:50:29.203536 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 491/971] 10.222.0.27 () {56 vars in 1120 bytes} [Fri Aug 5 19:50:29 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:50:29.210434 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df140fb-e90d-4639-93ea-1f4c5174aa78 None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:29.226534 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:29.325807 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df140fb-e90d-4639-93ea-1f4c5174aa78 None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:29.368426 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c83f2a-d8d4-42f8-bcbd-fb702d8ed4cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:29.369494 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c83f2a-d8d4-42f8-bcbd-fb702d8ed4cc None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:29.369774 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c83f2a-d8d4-42f8-bcbd-fb702d8ed4cc None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:29.370045 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c83f2a-d8d4-42f8-bcbd-fb702d8ed4cc None admin] PATH_INFO: `/v3/roles` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:29.370778 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c83f2a-d8d4-42f8-bcbd-fb702d8ed4cc None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:50:29.371084 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c83f2a-d8d4-42f8-bcbd-fb702d8ed4cc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:29.372530 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c83f2a-d8d4-42f8-bcbd-fb702d8ed4cc None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:29.386418 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 481/972] 10.222.0.27 () {56 vars in 1115 bytes} [Fri Aug 5 19:50:29 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:50:29.388078 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25fc364e-20a8-49af-8640-9f9f690b0276 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:29.498666 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25fc364e-20a8-49af-8640-9f9f690b0276 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:29.500234 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:29.538339 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f97885-8aff-4d94-ab7c-21395ff81564 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:29.538990 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f97885-8aff-4d94-ab7c-21395ff81564 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:29.539266 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f97885-8aff-4d94-ab7c-21395ff81564 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:29.539536 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f97885-8aff-4d94-ab7c-21395ff81564 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:29.552768 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f97885-8aff-4d94-ab7c-21395ff81564 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:29.554363 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f97885-8aff-4d94-ab7c-21395ff81564 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:29.571098 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-d3f97885-8aff-4d94-ab7c-21395ff81564 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:50:29.571098 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:50:29.571098 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:50:29.571098 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:50:29.571098 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:50:29.571098 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:50:29.571098 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:50:29.571098 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:50:29.571098 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:50:29.571098 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:50:29.571098 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:50:29.571098 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:50:29.571098 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 19:50:29.571098 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 19:50:29.571098 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 19:50:29.572054 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 19:50:29.572054 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:50:29.572054 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:50:29.572054 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:50:29.572054 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:50:29.572054 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:50:29.572054 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:50:29.572054 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:50:29.572054 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Aug 05 19:50:29.572054 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:50:29.572054 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:50:29.572054 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:50:29.572054 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:50:29.572054 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:50:29.572054 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:50:29.572054 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:50:29.572054 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:50:29.573110 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 19:50:29.573110 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 19:50:29.573110 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:50:29.573110 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Aug 05 19:50:29.573110 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 492/973] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Aug 5 19:50:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:50:29.578900 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c83f2a-d8d4-42f8-bcbd-fb702d8ed4cc None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:29.669406 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:29.695262 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c83f2a-d8d4-42f8-bcbd-fb702d8ed4cc None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:29.731255 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6340e94c-7b55-401f-ba85-1d77be4598b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:29.731880 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6340e94c-7b55-401f-ba85-1d77be4598b9 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:29.732162 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6340e94c-7b55-401f-ba85-1d77be4598b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:29.732431 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6340e94c-7b55-401f-ba85-1d77be4598b9 None admin] PATH_INFO: `/v3/domains` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:29.733161 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6340e94c-7b55-401f-ba85-1d77be4598b9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:50:29.733489 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6340e94c-7b55-401f-ba85-1d77be4598b9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:29.734919 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6340e94c-7b55-401f-ba85-1d77be4598b9 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:29.747688 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 482/974] 10.222.0.27 () {56 vars in 1109 bytes} [Fri Aug 5 19:50:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:50:29.753319 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f97885-8aff-4d94-ab7c-21395ff81564 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:29.859333 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f97885-8aff-4d94-ab7c-21395ff81564 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:29.894665 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdda198f-d85c-4d11-9dac-bb76f8c70958 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:29.895312 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdda198f-d85c-4d11-9dac-bb76f8c70958 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:29.895863 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdda198f-d85c-4d11-9dac-bb76f8c70958 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:29.895944 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7921b15-815b-4b8f-9946-b447eda4458c None None] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:29.896019 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdda198f-d85c-4d11-9dac-bb76f8c70958 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:29.896088 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7921b15-815b-4b8f-9946-b447eda4458c None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:29.896310 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7921b15-815b-4b8f-9946-b447eda4458c None None] PATH_INFO: `/` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:29.897901 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 483/975] 10.222.0.27 () {54 vars in 927 bytes} [Fri Aug 5 19:50:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:50:29.902977 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7358b584-fe1f-4909-a5cf-b8bd2e12c0b8 None None] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:29.903266 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7358b584-fe1f-4909-a5cf-b8bd2e12c0b8 None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:29.903546 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7358b584-fe1f-4909-a5cf-b8bd2e12c0b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:29.905631 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdda198f-d85c-4d11-9dac-bb76f8c70958 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:29.907124 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdda198f-d85c-4d11-9dac-bb76f8c70958 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:29.919458 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-bdda198f-d85c-4d11-9dac-bb76f8c70958 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 05 19:50:29.919458 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:50:29.919458 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:50:29.919458 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:50:29.919458 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:50:29.919458 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:50:29.919458 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:50:29.919458 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:50:29.919458 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:50:29.919458 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:50:29.919458 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:50:29.919458 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:50:29.919458 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 05 19:50:29.919458 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 05 19:50:29.919458 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 05 19:50:29.919458 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 05 19:50:29.920432 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:50:29.920432 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:50:29.920432 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 05 19:50:29.920432 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 19:50:29.920432 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 05 19:50:29.920432 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 19:50:29.920432 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:50:29.920432 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:50:29.920432 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:50:29.920432 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:50:29.920432 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:50:29.920432 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Aug 05 19:50:29.920432 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:50:29.920432 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:50:29.920432 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:50:29.920432 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:50:29.920432 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:50:29.920432 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:50:29.921415 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:50:29.921415 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:50:29.921415 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 05 19:50:29.921415 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 05 19:50:29.921415 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 05 19:50:29.921415 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 05 19:50:29.921415 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 05 19:50:29.921415 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 05 19:50:29.921415 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 05 19:50:29.921415 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Aug 05 19:50:29.924173 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.auth.core [None req-7358b584-fe1f-4909-a5cf-b8bd2e12c0b8 None None] MFA Rules not processed for user `03d1834a64cd4a43829827cb99a72747`. Rule list: `[]` (Enabled: `True`). {{(pid=80364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:50:29.924868 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 493/976] 10.222.0.27 () {56 vars in 1117 bytes} [Fri Aug 5 19:50:29 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:50:29.925858 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-7358b584-fe1f-4909-a5cf-b8bd2e12c0b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:50:29.926859 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdda198f-d85c-4d11-9dac-bb76f8c70958 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:29.947556 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:30.002993 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 484/977] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Aug 5 19:50:29 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 05 19:50:30.027211 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7358b584-fe1f-4909-a5cf-b8bd2e12c0b8 None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:30.040690 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdda198f-d85c-4d11-9dac-bb76f8c70958 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:30.077515 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7752ec47-86d4-487d-b06d-3d0f4b45daa8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:30.078139 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7752ec47-86d4-487d-b06d-3d0f4b45daa8 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:30.078420 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7752ec47-86d4-487d-b06d-3d0f4b45daa8 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:30.078693 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7752ec47-86d4-487d-b06d-3d0f4b45daa8 None admin] PATH_INFO: `/v3/users` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:30.079496 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7752ec47-86d4-487d-b06d-3d0f4b45daa8 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:50:30.079806 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7752ec47-86d4-487d-b06d-3d0f4b45daa8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:30.081273 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7752ec47-86d4-487d-b06d-3d0f4b45daa8 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:30.099782 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 494/978] 10.222.0.27 () {56 vars in 1149 bytes} [Fri Aug 5 19:50:29 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:50:30.105247 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7752ec47-86d4-487d-b06d-3d0f4b45daa8 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:30.106355 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:30.145076 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7358b584-fe1f-4909-a5cf-b8bd2e12c0b8 None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:30.183396 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6859afc0-9b35-4166-bb1c-caac0de3a2dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:30.184079 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6859afc0-9b35-4166-bb1c-caac0de3a2dc None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:30.184358 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6859afc0-9b35-4166-bb1c-caac0de3a2dc None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:30.184680 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6859afc0-9b35-4166-bb1c-caac0de3a2dc None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:30.185311 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6859afc0-9b35-4166-bb1c-caac0de3a2dc None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:30.185926 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:30.186765 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6859afc0-9b35-4166-bb1c-caac0de3a2dc None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:30.196071 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application [None req-6859afc0-9b35-4166-bb1c-caac0de3a2dc None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 05 19:50:30.196071 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:50:30.196071 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 05 19:50:30.196071 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 05 19:50:30.196071 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 05 19:50:30.196071 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 05 19:50:30.196071 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 05 19:50:30.196071 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 05 19:50:30.196071 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 05 19:50:30.196071 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Aug 05 19:50:30.196071 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 05 19:50:30.196071 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Aug 05 19:50:30.196071 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:50:30.196071 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:50:30.196071 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:50:30.196071 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:50:30.196071 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:50:30.197229 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:50:30.197229 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:50:30.197229 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:50:30.197229 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:50:30.197229 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:50:30.197229 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:50:30.197229 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 05 19:50:30.197229 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 05 19:50:30.197229 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 05 19:50:30.197229 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 05 19:50:30.197229 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:50:30.197229 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:50:30.197229 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:50:30.197229 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:50:30.197229 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:50:30.197229 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:50:30.198087 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:50:30.198087 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application with Lock( Aug 05 19:50:30.198087 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:50:30.198087 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:50:30.198087 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:50:30.198087 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:50:30.198087 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:50:30.198087 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:50:30.198087 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:50:30.198087 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:50:30.198087 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 05 19:50:30.198087 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 05 19:50:30.198087 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 05 19:50:30.198087 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Aug 05 19:50:30.198087 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 485/979] 10.222.0.27 () {56 vars in 1126 bytes} [Fri Aug 5 19:50:30 2022] GET /identity/v3/services/object-store => generated 93 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 05 19:50:30.203473 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6859afc0-9b35-4166-bb1c-caac0de3a2dc None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:30.224477 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7752ec47-86d4-487d-b06d-3d0f4b45daa8 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:30.271091 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913fdbfd-1647-4190-ad34-587d363c593d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:30.271722 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913fdbfd-1647-4190-ad34-587d363c593d None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:30.272007 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913fdbfd-1647-4190-ad34-587d363c593d None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:30.272287 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913fdbfd-1647-4190-ad34-587d363c593d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:30.321286 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6859afc0-9b35-4166-bb1c-caac0de3a2dc None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:30.353932 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:30.360037 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913fdbfd-1647-4190-ad34-587d363c593d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:30.360037 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913fdbfd-1647-4190-ad34-587d363c593d None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:30.360352 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f2cfa6-96e4-48e4-a365-6ee6a273445f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:30.360934 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f2cfa6-96e4-48e4-a365-6ee6a273445f None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:30.361277 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f2cfa6-96e4-48e4-a365-6ee6a273445f None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:30.361687 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f2cfa6-96e4-48e4-a365-6ee6a273445f None admin] PATH_INFO: `/v3/services` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:30.362226 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f2cfa6-96e4-48e4-a365-6ee6a273445f None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:50:30.362531 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f2cfa6-96e4-48e4-a365-6ee6a273445f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:30.366739 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f2cfa6-96e4-48e4-a365-6ee6a273445f None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:30.377876 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 486/980] 10.222.0.27 () {56 vars in 1122 bytes} [Fri Aug 5 19:50:30 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:50:30.378739 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f2cfa6-96e4-48e4-a365-6ee6a273445f None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:30.432794 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:30.465429 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 495/981] 10.222.0.27 () {58 vars in 1282 bytes} [Fri Aug 5 19:50:30 2022] GET /identity/v3/auth/tokens => generated 1570 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 05 19:50:30.467056 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5b29bc-c204-440a-a17b-967312149bfe None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:30.467345 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5b29bc-c204-440a-a17b-967312149bfe None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:30.467618 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5b29bc-c204-440a-a17b-967312149bfe None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:30.469164 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 496/982] 10.222.0.27 () {54 vars in 927 bytes} [Fri Aug 5 19:50:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:50:30.471094 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5b29bc-c204-440a-a17b-967312149bfe None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:30.483876 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f2cfa6-96e4-48e4-a365-6ee6a273445f None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:30.519778 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56d7a8d-e13d-4e43-845c-0d204bb4a743 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:30.520710 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56d7a8d-e13d-4e43-845c-0d204bb4a743 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:30.520999 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56d7a8d-e13d-4e43-845c-0d204bb4a743 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:30.521279 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56d7a8d-e13d-4e43-845c-0d204bb4a743 None admin] PATH_INFO: `/v3/services` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:30.521984 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b56d7a8d-e13d-4e43-845c-0d204bb4a743 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:50:30.522465 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b56d7a8d-e13d-4e43-845c-0d204bb4a743 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:30.524663 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b56d7a8d-e13d-4e43-845c-0d204bb4a743 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:30.530177 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 487/983] 10.222.0.27 () {56 vars in 1122 bytes} [Fri Aug 5 19:50:30 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:50:30.531561 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17703960-5384-4663-aeb1-81d5daed5f66 None None] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:30.531688 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17703960-5384-4663-aeb1-81d5daed5f66 None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:30.531966 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17703960-5384-4663-aeb1-81d5daed5f66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:30.550206 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.auth.core [None req-17703960-5384-4663-aeb1-81d5daed5f66 None None] MFA Rules not processed for user `03d1834a64cd4a43829827cb99a72747`. Rule list: `[]` (Enabled: `True`). {{(pid=80364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:50:30.551422 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-17703960-5384-4663-aeb1-81d5daed5f66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:50:30.578911 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5b29bc-c204-440a-a17b-967312149bfe None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:30.609007 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:30.617916 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b991630-230f-45bb-af48-76f1bde2842b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:30.618568 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b991630-230f-45bb-af48-76f1bde2842b None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:30.618851 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b991630-230f-45bb-af48-76f1bde2842b None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:30.619143 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b991630-230f-45bb-af48-76f1bde2842b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:30.628123 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 488/984] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Aug 5 19:50:30 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 05 19:50:30.629117 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17703960-5384-4663-aeb1-81d5daed5f66 None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:30.630781 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b991630-230f-45bb-af48-76f1bde2842b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:30.632255 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b991630-230f-45bb-af48-76f1bde2842b None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:30.648698 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-9b991630-230f-45bb-af48-76f1bde2842b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:50:30.648698 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:50:30.648698 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:50:30.648698 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:50:30.648698 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:50:30.648698 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:50:30.648698 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:50:30.648698 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:50:30.648698 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:50:30.648698 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:50:30.648698 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:50:30.648698 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:50:30.648698 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 19:50:30.648698 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 19:50:30.648698 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 19:50:30.649958 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 19:50:30.649958 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:50:30.649958 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:50:30.649958 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:50:30.649958 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:50:30.649958 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:50:30.649958 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:50:30.649958 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:50:30.649958 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Aug 05 19:50:30.649958 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:50:30.649958 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:50:30.649958 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:50:30.649958 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:50:30.649958 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:50:30.649958 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:50:30.649958 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:50:30.649958 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:50:30.651731 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 19:50:30.651731 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 19:50:30.651731 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:50:30.651731 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Aug 05 19:50:30.651731 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 497/985] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Aug 5 19:50:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:50:30.651731 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b991630-230f-45bb-af48-76f1bde2842b None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:30.681394 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:30.736849 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17703960-5384-4663-aeb1-81d5daed5f66 None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:30.764807 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b991630-230f-45bb-af48-76f1bde2842b None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:30.773206 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da51b68-25b9-4f1b-8046-255ca6f5ab4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:30.773844 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da51b68-25b9-4f1b-8046-255ca6f5ab4c None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:30.774140 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da51b68-25b9-4f1b-8046-255ca6f5ab4c None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:30.774669 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da51b68-25b9-4f1b-8046-255ca6f5ab4c None admin] PATH_INFO: `/v3/endpoints` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:30.775452 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da51b68-25b9-4f1b-8046-255ca6f5ab4c None admin] RBAC: Adding query filter params (interface=admin, service_id=8b4ed76680e84a3cbd78a0a9877d5608, region_id=RegionOne) {{(pid=80364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:50:30.775763 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da51b68-25b9-4f1b-8046-255ca6f5ab4c None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:30.777165 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da51b68-25b9-4f1b-8046-255ca6f5ab4c None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:30.787232 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 489/986] 10.222.0.27 () {56 vars in 1249 bytes} [Fri Aug 5 19:50:30 2022] GET /identity/v3/endpoints?service_id=8b4ed76680e84a3cbd78a0a9877d5608&interface=admin®ion_id=RegionOne => generated 193 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:50:30.787232 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da51b68-25b9-4f1b-8046-255ca6f5ab4c None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:30.800693 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b40b12-5371-4a43-9083-d3812deb3f63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:30.801349 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b40b12-5371-4a43-9083-d3812deb3f63 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:30.801634 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b40b12-5371-4a43-9083-d3812deb3f63 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:30.801912 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b40b12-5371-4a43-9083-d3812deb3f63 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:30.802555 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b40b12-5371-4a43-9083-d3812deb3f63 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:30.803942 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b40b12-5371-4a43-9083-d3812deb3f63 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:30.813815 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-c4b40b12-5371-4a43-9083-d3812deb3f63 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 05 19:50:30.813815 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:50:30.813815 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 05 19:50:30.813815 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 05 19:50:30.813815 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 05 19:50:30.813815 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 05 19:50:30.813815 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 05 19:50:30.813815 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 05 19:50:30.813815 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 05 19:50:30.813815 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Aug 05 19:50:30.813815 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 05 19:50:30.813815 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Aug 05 19:50:30.813815 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:50:30.813815 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:50:30.813815 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:50:30.813815 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:50:30.813815 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:50:30.814965 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:50:30.814965 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:50:30.814965 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:50:30.814965 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:50:30.814965 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:50:30.814965 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:50:30.814965 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 05 19:50:30.814965 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 05 19:50:30.814965 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 05 19:50:30.814965 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 05 19:50:30.814965 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:50:30.814965 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:50:30.814965 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:50:30.814965 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:50:30.814965 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:50:30.814965 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:50:30.815810 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:50:30.815810 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Aug 05 19:50:30.815810 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:50:30.815810 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:50:30.815810 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:50:30.815810 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:50:30.815810 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:50:30.815810 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:50:30.815810 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:50:30.815810 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:50:30.815810 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 05 19:50:30.815810 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 05 19:50:30.815810 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 05 19:50:30.815810 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Aug 05 19:50:30.815810 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 498/987] 10.222.0.27 () {56 vars in 1111 bytes} [Fri Aug 5 19:50:30 2022] GET /identity/v3/services/compute => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 05 19:50:30.816632 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b40b12-5371-4a43-9083-d3812deb3f63 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:30.846456 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:30.894412 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da51b68-25b9-4f1b-8046-255ca6f5ab4c None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:30.909460 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:30.922774 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b40b12-5371-4a43-9083-d3812deb3f63 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:30.931348 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19d0502-115b-4aa2-8d45-9e51f896220a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:30.931972 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19d0502-115b-4aa2-8d45-9e51f896220a None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:30.932249 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19d0502-115b-4aa2-8d45-9e51f896220a None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:30.932551 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19d0502-115b-4aa2-8d45-9e51f896220a None admin] PATH_INFO: `/v3/domains` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:30.933363 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d19d0502-115b-4aa2-8d45-9e51f896220a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:50:30.933673 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d19d0502-115b-4aa2-8d45-9e51f896220a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:30.935066 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d19d0502-115b-4aa2-8d45-9e51f896220a None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:30.946506 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 490/988] 10.222.0.27 () {56 vars in 1109 bytes} [Fri Aug 5 19:50:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:50:30.947333 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19d0502-115b-4aa2-8d45-9e51f896220a None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:30.957385 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376fc500-32bb-45c9-8a3a-b8ab63ccca99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:30.958009 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376fc500-32bb-45c9-8a3a-b8ab63ccca99 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:30.958292 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376fc500-32bb-45c9-8a3a-b8ab63ccca99 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:30.958565 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376fc500-32bb-45c9-8a3a-b8ab63ccca99 None admin] PATH_INFO: `/v3/services` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:30.959225 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-376fc500-32bb-45c9-8a3a-b8ab63ccca99 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:30.960647 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-376fc500-32bb-45c9-8a3a-b8ab63ccca99 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:30.967762 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 499/989] 10.222.0.27 () {56 vars in 1087 bytes} [Fri Aug 5 19:50:30 2022] GET /identity/v3/services => generated 1727 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 19:50:30.968617 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376fc500-32bb-45c9-8a3a-b8ab63ccca99 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:31.081231 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19d0502-115b-4aa2-8d45-9e51f896220a None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:31.093463 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376fc500-32bb-45c9-8a3a-b8ab63ccca99 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:31.096880 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:31.122773 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf0e6c0-6f41-44d4-abb0-d246d2bef8db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:31.123418 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf0e6c0-6f41-44d4-abb0-d246d2bef8db None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:31.123701 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf0e6c0-6f41-44d4-abb0-d246d2bef8db None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:31.123973 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf0e6c0-6f41-44d4-abb0-d246d2bef8db None admin] PATH_INFO: `/v3/services` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:31.124704 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf0e6c0-6f41-44d4-abb0-d246d2bef8db None admin] RBAC: Adding query filter params (name=compute) {{(pid=80364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:50:31.125007 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf0e6c0-6f41-44d4-abb0-d246d2bef8db None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:31.126411 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf0e6c0-6f41-44d4-abb0-d246d2bef8db None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:31.132307 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 491/990] 10.222.0.27 () {56 vars in 1112 bytes} [Fri Aug 5 19:50:30 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:50:31.138314 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf0e6c0-6f41-44d4-abb0-d246d2bef8db None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:31.139881 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c064ee-041e-4a5d-be0d-b194c566df5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:31.140568 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c064ee-041e-4a5d-be0d-b194c566df5d None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:31.141436 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c064ee-041e-4a5d-be0d-b194c566df5d None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:31.141436 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c064ee-041e-4a5d-be0d-b194c566df5d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:31.150589 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c064ee-041e-4a5d-be0d-b194c566df5d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:31.152062 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c064ee-041e-4a5d-be0d-b194c566df5d None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:31.165439 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-d1c064ee-041e-4a5d-be0d-b194c566df5d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 19:50:31.165439 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:50:31.165439 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:50:31.165439 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:50:31.165439 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:50:31.165439 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:50:31.165439 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:50:31.165439 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:50:31.165439 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:50:31.165439 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:50:31.165439 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:50:31.165439 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:50:31.165439 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 05 19:50:31.165439 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 05 19:50:31.165439 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 05 19:50:31.166399 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 05 19:50:31.166399 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:50:31.166399 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:50:31.166399 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:50:31.166399 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:50:31.166399 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:50:31.166399 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:50:31.166399 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:50:31.166399 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Aug 05 19:50:31.166399 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:50:31.166399 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:50:31.166399 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:50:31.166399 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:50:31.166399 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:50:31.166399 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:50:31.166399 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:50:31.166399 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:50:31.167306 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 05 19:50:31.167306 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 05 19:50:31.167306 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 05 19:50:31.167306 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 05 19:50:31.167306 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 05 19:50:31.167306 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 05 19:50:31.167306 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 19:50:31.167306 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Aug 05 19:50:31.167306 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 500/991] 10.222.0.27 () {56 vars in 1111 bytes} [Fri Aug 5 19:50:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:50:31.173322 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c064ee-041e-4a5d-be0d-b194c566df5d None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:31.190634 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:31.256682 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf0e6c0-6f41-44d4-abb0-d246d2bef8db None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:31.285101 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c064ee-041e-4a5d-be0d-b194c566df5d None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:31.296095 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3843caa-d01f-4db4-827b-7c73caaec23a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:31.297114 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3843caa-d01f-4db4-827b-7c73caaec23a None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:31.297406 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3843caa-d01f-4db4-827b-7c73caaec23a None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:31.297852 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3843caa-d01f-4db4-827b-7c73caaec23a None admin] PATH_INFO: `/v3/services` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:31.298557 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3843caa-d01f-4db4-827b-7c73caaec23a None admin] RBAC: Adding query filter params (type=compute) {{(pid=80364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:50:31.298865 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3843caa-d01f-4db4-827b-7c73caaec23a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:31.300309 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3843caa-d01f-4db4-827b-7c73caaec23a None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:31.307246 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 492/992] 10.222.0.27 () {56 vars in 1112 bytes} [Fri Aug 5 19:50:31 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 19:50:31.313425 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3843caa-d01f-4db4-827b-7c73caaec23a None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:31.327820 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9416eb43-00c1-4fb7-99f2-97a054538e8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:31.328529 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9416eb43-00c1-4fb7-99f2-97a054538e8d None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:31.328951 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9416eb43-00c1-4fb7-99f2-97a054538e8d None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:31.329251 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9416eb43-00c1-4fb7-99f2-97a054538e8d None admin] PATH_INFO: `/v3/projects` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:31.330118 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9416eb43-00c1-4fb7-99f2-97a054538e8d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:50:31.330427 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9416eb43-00c1-4fb7-99f2-97a054538e8d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:31.331902 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9416eb43-00c1-4fb7-99f2-97a054538e8d None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:31.348943 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 501/993] 10.222.0.27 () {56 vars in 1148 bytes} [Fri Aug 5 19:50:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 19:50:31.354488 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9416eb43-00c1-4fb7-99f2-97a054538e8d None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:31.364609 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:31.435004 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3843caa-d01f-4db4-827b-7c73caaec23a None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:31.437477 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:31.474416 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9416eb43-00c1-4fb7-99f2-97a054538e8d None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:31.479012 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3050e06f-8658-43d0-8fd5-230cc6bd5419 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:31.479643 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3050e06f-8658-43d0-8fd5-230cc6bd5419 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:31.479920 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3050e06f-8658-43d0-8fd5-230cc6bd5419 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:31.480188 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3050e06f-8658-43d0-8fd5-230cc6bd5419 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:31.481577 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3050e06f-8658-43d0-8fd5-230cc6bd5419 None admin] RBAC: Adding query filter params (interface=public, service_id=a0c15ae1c9af4030bad1020f495b0393, region_id=RegionOne) {{(pid=80364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:50:31.481885 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3050e06f-8658-43d0-8fd5-230cc6bd5419 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:31.483266 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3050e06f-8658-43d0-8fd5-230cc6bd5419 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:31.492767 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 493/994] 10.222.0.27 () {56 vars in 1251 bytes} [Fri Aug 5 19:50:31 2022] GET /identity/v3/endpoints?service_id=a0c15ae1c9af4030bad1020f495b0393&interface=public®ion_id=RegionOne => generated 194 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 19:50:31.498299 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3050e06f-8658-43d0-8fd5-230cc6bd5419 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:31.519035 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1d5a1e-1dd6-4426-9869-71f5aa7ce8cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:31.519947 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1d5a1e-1dd6-4426-9869-71f5aa7ce8cf None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:31.520394 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1d5a1e-1dd6-4426-9869-71f5aa7ce8cf None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:31.520837 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1d5a1e-1dd6-4426-9869-71f5aa7ce8cf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:31.521806 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b1d5a1e-1dd6-4426-9869-71f5aa7ce8cf None admin] RBAC: Adding query filter params (role.id=31647643dd7c43aba270f63d5fe83acb, scope.project.id=a70a82dcfc574fad90b3f5f00abbba58, user.id=bdf7cf05a77e4298b2bf4f58b40f54aa) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:50:31.522550 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b1d5a1e-1dd6-4426-9869-71f5aa7ce8cf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:31.523872 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b1d5a1e-1dd6-4426-9869-71f5aa7ce8cf None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:31.540567 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 502/995] 10.222.0.27 () {56 vars in 1374 bytes} [Fri Aug 5 19:50:31 2022] GET /identity/v3/role_assignments?user.id=bdf7cf05a77e4298b2bf4f58b40f54aa&scope.project.id=a70a82dcfc574fad90b3f5f00abbba58&role.id=31647643dd7c43aba270f63d5fe83acb => generated 608 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 19:50:31.625799 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3050e06f-8658-43d0-8fd5-230cc6bd5419 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:31.628802 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:31.667468 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9e7b80-5070-45af-bffd-c393398f0b1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:31.668178 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9e7b80-5070-45af-bffd-c393398f0b1e None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:31.668460 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9e7b80-5070-45af-bffd-c393398f0b1e None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:31.668764 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9e7b80-5070-45af-bffd-c393398f0b1e None admin] PATH_INFO: `/v3/services` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:31.669443 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e9e7b80-5070-45af-bffd-c393398f0b1e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:31.670883 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e9e7b80-5070-45af-bffd-c393398f0b1e None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:31.678224 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 494/996] 10.222.0.27 () {56 vars in 1087 bytes} [Fri Aug 5 19:50:31 2022] GET /identity/v3/services => generated 1727 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 05 19:50:33.030266 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7072f22f-d0ec-4075-bb4c-9100e60c2a07 None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:33.030793 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7072f22f-d0ec-4075-bb4c-9100e60c2a07 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:33.030793 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7072f22f-d0ec-4075-bb4c-9100e60c2a07 None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:33.032306 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 503/997] 10.222.0.27 () {54 vars in 927 bytes} [Fri Aug 5 19:50:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:50:33.037283 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8364ca-d61a-47c1-94a4-c56887e160c4 None None] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:33.037740 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8364ca-d61a-47c1-94a4-c56887e160c4 None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:33.037861 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8364ca-d61a-47c1-94a4-c56887e160c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:33.055581 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.auth.core [None req-9c8364ca-d61a-47c1-94a4-c56887e160c4 None None] MFA Rules not processed for user `03d1834a64cd4a43829827cb99a72747`. Rule list: `[]` (Enabled: `True`). {{(pid=80364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:50:33.056789 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-9c8364ca-d61a-47c1-94a4-c56887e160c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:50:33.123765 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 495/998] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Aug 5 19:50:33 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 05 19:50:33.146461 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7072f22f-d0ec-4075-bb4c-9100e60c2a07 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:33.248547 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7072f22f-d0ec-4075-bb4c-9100e60c2a07 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:33.287911 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b8fa0a-781a-4969-8e8f-32c5427831bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:33.288471 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b8fa0a-781a-4969-8e8f-32c5427831bf None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:33.288782 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b8fa0a-781a-4969-8e8f-32c5427831bf None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:33.289053 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b8fa0a-781a-4969-8e8f-32c5427831bf None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:33.289695 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b8fa0a-781a-4969-8e8f-32c5427831bf None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:33.291075 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b8fa0a-781a-4969-8e8f-32c5427831bf None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:33.294228 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:33.301592 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-b9b8fa0a-781a-4969-8e8f-32c5427831bf None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 05 19:50:33.301592 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:50:33.301592 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 05 19:50:33.301592 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 05 19:50:33.301592 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 05 19:50:33.301592 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 05 19:50:33.301592 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 05 19:50:33.301592 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 05 19:50:33.301592 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 05 19:50:33.301592 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Aug 05 19:50:33.301592 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 05 19:50:33.301592 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Aug 05 19:50:33.301592 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:50:33.301592 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:50:33.301592 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:50:33.301592 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:50:33.301592 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:50:33.302716 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:50:33.302716 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:50:33.302716 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:50:33.302716 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:50:33.302716 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:50:33.302716 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:50:33.302716 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 05 19:50:33.302716 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 05 19:50:33.302716 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 05 19:50:33.302716 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 05 19:50:33.302716 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:50:33.302716 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:50:33.302716 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:50:33.302716 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:50:33.302716 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:50:33.302716 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:50:33.303572 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:50:33.303572 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Aug 05 19:50:33.303572 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:50:33.303572 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:50:33.303572 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:50:33.303572 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:50:33.303572 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:50:33.303572 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:50:33.303572 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:50:33.303572 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:50:33.303572 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 05 19:50:33.303572 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 05 19:50:33.303572 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 05 19:50:33.303572 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Aug 05 19:50:33.310666 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 504/999] 10.222.0.27 () {56 vars in 1126 bytes} [Fri Aug 5 19:50:33 2022] GET /identity/v3/services/object-store => generated 93 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:50:33.310839 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8364ca-d61a-47c1-94a4-c56887e160c4 None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:33.417036 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:33.421100 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8364ca-d61a-47c1-94a4-c56887e160c4 None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:33.456398 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c7fc46-11f8-4d39-b3a6-3166f6d51fde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:33.457047 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c7fc46-11f8-4d39-b3a6-3166f6d51fde None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:33.457326 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c7fc46-11f8-4d39-b3a6-3166f6d51fde None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:33.457600 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c7fc46-11f8-4d39-b3a6-3166f6d51fde None admin] PATH_INFO: `/v3/services` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:33.458277 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c7fc46-11f8-4d39-b3a6-3166f6d51fde None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:50:33.458580 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c7fc46-11f8-4d39-b3a6-3166f6d51fde None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:33.460063 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c7fc46-11f8-4d39-b3a6-3166f6d51fde None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:33.466074 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 496/1000] 10.222.0.27 () {56 vars in 1122 bytes} [Fri Aug 5 19:50:33 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:50:33.471528 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b8fa0a-781a-4969-8e8f-32c5427831bf None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:33.572801 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b8fa0a-781a-4969-8e8f-32c5427831bf None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:33.601473 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20290418-8474-44bc-af01-02610f8cb210 None None] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:33.601761 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20290418-8474-44bc-af01-02610f8cb210 None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:33.602035 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20290418-8474-44bc-af01-02610f8cb210 None None] PATH_INFO: `/` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:33.603619 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 497/1001] 10.222.0.27 () {54 vars in 927 bytes} [Fri Aug 5 19:50:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 05 19:50:33.608419 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b806d8c5-4872-44a2-9d7f-e912067b7e6d None None] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:33.608662 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39de9d5d-1d82-4b69-9635-ef8004888745 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:33.608790 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b806d8c5-4872-44a2-9d7f-e912067b7e6d None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:33.608995 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b806d8c5-4872-44a2-9d7f-e912067b7e6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:33.609195 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39de9d5d-1d82-4b69-9635-ef8004888745 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:33.609476 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39de9d5d-1d82-4b69-9635-ef8004888745 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:33.609749 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39de9d5d-1d82-4b69-9635-ef8004888745 None admin] PATH_INFO: `/v3/services` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:33.610437 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39de9d5d-1d82-4b69-9635-ef8004888745 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:50:33.610743 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39de9d5d-1d82-4b69-9635-ef8004888745 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:33.612102 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39de9d5d-1d82-4b69-9635-ef8004888745 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:33.617513 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 505/1002] 10.222.0.27 () {56 vars in 1122 bytes} [Fri Aug 5 19:50:33 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:50:33.622955 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39de9d5d-1d82-4b69-9635-ef8004888745 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:33.627245 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.auth.core [None req-b806d8c5-4872-44a2-9d7f-e912067b7e6d None None] MFA Rules not processed for user `03d1834a64cd4a43829827cb99a72747`. Rule list: `[]` (Enabled: `True`). {{(pid=80364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:50:33.628421 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-b806d8c5-4872-44a2-9d7f-e912067b7e6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:50:33.691164 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:33.703311 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 498/1003] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Aug 5 19:50:33 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 05 19:50:33.724910 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b806d8c5-4872-44a2-9d7f-e912067b7e6d None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:33.733627 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39de9d5d-1d82-4b69-9635-ef8004888745 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:33.767471 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d580486b-81b1-4eb2-bc28-81e5c8874dba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:33.768108 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d580486b-81b1-4eb2-bc28-81e5c8874dba None admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:33.768383 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d580486b-81b1-4eb2-bc28-81e5c8874dba None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:33.768680 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d580486b-81b1-4eb2-bc28-81e5c8874dba None admin] PATH_INFO: `/v3/endpoints` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:33.769382 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d580486b-81b1-4eb2-bc28-81e5c8874dba None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:33.770735 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d580486b-81b1-4eb2-bc28-81e5c8874dba None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:33.787360 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 506/1004] 10.222.0.27 () {60 vars in 1144 bytes} [Fri Aug 5 19:50:33 2022] POST /identity/v3/endpoints => generated 325 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 05 19:50:33.788844 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f40d1d5-8f9c-40e1-b450-34ca7f956ba3 None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:33.789325 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f40d1d5-8f9c-40e1-b450-34ca7f956ba3 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:33.789745 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f40d1d5-8f9c-40e1-b450-34ca7f956ba3 None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:33.791187 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 507/1005] 10.222.0.27 () {54 vars in 927 bytes} [Fri Aug 5 19:50:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:50:33.801721 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5b53da-3882-418b-8023-36832caf86ca None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:33.802050 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5b53da-3882-418b-8023-36832caf86ca None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:33.802402 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5b53da-3882-418b-8023-36832caf86ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:33.819521 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:33.827566 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-9e5b53da-3882-418b-8023-36832caf86ca None None] MFA Rules not processed for user `03d1834a64cd4a43829827cb99a72747`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:50:33.828876 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-9e5b53da-3882-418b-8023-36832caf86ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:50:33.830949 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b806d8c5-4872-44a2-9d7f-e912067b7e6d None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:33.877238 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b53fc5d-8366-44dd-9e43-3f15f8d89b43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:33.877891 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b53fc5d-8366-44dd-9e43-3f15f8d89b43 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:33.878168 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b53fc5d-8366-44dd-9e43-3f15f8d89b43 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:33.878437 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b53fc5d-8366-44dd-9e43-3f15f8d89b43 None admin] PATH_INFO: `/v3/services/image` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:33.879082 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b53fc5d-8366-44dd-9e43-3f15f8d89b43 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:33.880640 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b53fc5d-8366-44dd-9e43-3f15f8d89b43 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:33.890623 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application [None req-3b53fc5d-8366-44dd-9e43-3f15f8d89b43 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 05 19:50:33.890623 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:50:33.890623 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 05 19:50:33.890623 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 05 19:50:33.890623 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 05 19:50:33.890623 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 05 19:50:33.890623 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 05 19:50:33.890623 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 05 19:50:33.890623 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 05 19:50:33.890623 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Aug 05 19:50:33.890623 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 05 19:50:33.890623 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Aug 05 19:50:33.890623 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:50:33.890623 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:50:33.890623 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:50:33.890623 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:50:33.890623 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:50:33.891716 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:50:33.891716 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:50:33.891716 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:50:33.891716 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:50:33.891716 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:50:33.891716 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:50:33.891716 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 05 19:50:33.891716 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 05 19:50:33.891716 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 05 19:50:33.891716 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 05 19:50:33.891716 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:50:33.891716 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:50:33.891716 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:50:33.891716 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:50:33.891716 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:50:33.891716 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:50:33.892633 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:50:33.892633 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application with Lock( Aug 05 19:50:33.892633 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:50:33.892633 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:50:33.892633 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:50:33.892633 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:50:33.892633 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:50:33.892633 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:50:33.892633 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:50:33.892633 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:50:33.892633 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 05 19:50:33.892633 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 05 19:50:33.892633 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 05 19:50:33.892633 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Aug 05 19:50:33.892633 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 499/1006] 10.222.0.27 () {56 vars in 1105 bytes} [Fri Aug 5 19:50:33 2022] GET /identity/v3/services/image => generated 86 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 05 19:50:33.897859 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b53fc5d-8366-44dd-9e43-3f15f8d89b43 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:33.914392 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 508/1007] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Aug 5 19:50:33 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 05 19:50:33.939022 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5b53da-3882-418b-8023-36832caf86ca None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:34.003480 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:34.038679 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b53fc5d-8366-44dd-9e43-3f15f8d89b43 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:34.063553 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5b53da-3882-418b-8023-36832caf86ca None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:34.085628 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504ff1e1-2c97-4088-8a19-ec07d9870a5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:34.086267 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504ff1e1-2c97-4088-8a19-ec07d9870a5f None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:34.086540 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504ff1e1-2c97-4088-8a19-ec07d9870a5f None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:34.086815 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504ff1e1-2c97-4088-8a19-ec07d9870a5f None admin] PATH_INFO: `/v3/services` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:34.087503 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-504ff1e1-2c97-4088-8a19-ec07d9870a5f None admin] RBAC: Adding query filter params (name=image) {{(pid=80364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:50:34.087810 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-504ff1e1-2c97-4088-8a19-ec07d9870a5f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:34.089221 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-504ff1e1-2c97-4088-8a19-ec07d9870a5f None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:34.096545 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 500/1008] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Aug 5 19:50:33 2022] GET /identity/v3/services?name=image => generated 122 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 19:50:34.100144 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504ff1e1-2c97-4088-8a19-ec07d9870a5f None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:34.103313 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2bfe073-34c7-4a70-8db2-3f2a24fc9f93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:34.103975 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bfe073-34c7-4a70-8db2-3f2a24fc9f93 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:34.104255 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bfe073-34c7-4a70-8db2-3f2a24fc9f93 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:34.104547 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bfe073-34c7-4a70-8db2-3f2a24fc9f93 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:34.105188 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2bfe073-34c7-4a70-8db2-3f2a24fc9f93 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:34.107098 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2bfe073-34c7-4a70-8db2-3f2a24fc9f93 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:34.116715 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-c2bfe073-34c7-4a70-8db2-3f2a24fc9f93 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 05 19:50:34.116715 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:50:34.116715 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 05 19:50:34.116715 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 05 19:50:34.116715 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 05 19:50:34.116715 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 05 19:50:34.116715 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 05 19:50:34.116715 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 05 19:50:34.116715 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 05 19:50:34.116715 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Aug 05 19:50:34.116715 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 05 19:50:34.116715 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Aug 05 19:50:34.116715 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:50:34.116715 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:50:34.116715 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:50:34.116715 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:50:34.116715 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:50:34.117895 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:50:34.117895 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:50:34.117895 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:50:34.117895 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:50:34.117895 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:50:34.117895 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:50:34.117895 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 05 19:50:34.117895 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 05 19:50:34.117895 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 05 19:50:34.117895 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 05 19:50:34.117895 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:50:34.117895 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:50:34.117895 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:50:34.117895 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:50:34.117895 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:50:34.117895 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:50:34.118780 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:50:34.118780 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Aug 05 19:50:34.118780 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:50:34.118780 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:50:34.118780 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:50:34.118780 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:50:34.118780 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:50:34.118780 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:50:34.118780 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:50:34.118780 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:50:34.118780 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 05 19:50:34.118780 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 05 19:50:34.118780 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 05 19:50:34.118780 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Aug 05 19:50:34.118780 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 509/1009] 10.222.0.27 () {56 vars in 1111 bytes} [Fri Aug 5 19:50:33 2022] GET /identity/v3/services/compute => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 05 19:50:34.124693 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:34.125495 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2bfe073-34c7-4a70-8db2-3f2a24fc9f93 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:34.231771 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504ff1e1-2c97-4088-8a19-ec07d9870a5f None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:34.276237 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2bfe073-34c7-4a70-8db2-3f2a24fc9f93 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:34.301721 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:34.311723 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9ffa9a-77d9-4a3c-bc15-7d0255edc826 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:34.313224 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9ffa9a-77d9-4a3c-bc15-7d0255edc826 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:34.313846 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9ffa9a-77d9-4a3c-bc15-7d0255edc826 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:34.314124 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9ffa9a-77d9-4a3c-bc15-7d0255edc826 None admin] PATH_INFO: `/v3/services` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:34.315003 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9ffa9a-77d9-4a3c-bc15-7d0255edc826 None admin] RBAC: Adding query filter params (type=image) {{(pid=80364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:50:34.315315 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9ffa9a-77d9-4a3c-bc15-7d0255edc826 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:34.316842 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9ffa9a-77d9-4a3c-bc15-7d0255edc826 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:34.326428 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 501/1010] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Aug 5 19:50:34 2022] GET /identity/v3/services?type=image => generated 122 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 19:50:34.333307 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec716fa-b343-418e-8d3a-537d4d8707bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:34.334491 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec716fa-b343-418e-8d3a-537d4d8707bc None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:34.334806 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec716fa-b343-418e-8d3a-537d4d8707bc None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:34.335281 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec716fa-b343-418e-8d3a-537d4d8707bc None admin] PATH_INFO: `/v3/services` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:34.336779 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec716fa-b343-418e-8d3a-537d4d8707bc None admin] RBAC: Adding query filter params (name=compute) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:50:34.336955 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec716fa-b343-418e-8d3a-537d4d8707bc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:34.338504 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec716fa-b343-418e-8d3a-537d4d8707bc None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:34.345320 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 510/1011] 10.222.0.27 () {56 vars in 1112 bytes} [Fri Aug 5 19:50:34 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:50:34.355154 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9ffa9a-77d9-4a3c-bc15-7d0255edc826 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:34.477734 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:34.500264 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9ffa9a-77d9-4a3c-bc15-7d0255edc826 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:34.548221 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45378486-fd12-4a10-85f8-9b396079ba3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:34.548221 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45378486-fd12-4a10-85f8-9b396079ba3e None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:34.548221 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45378486-fd12-4a10-85f8-9b396079ba3e None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:34.548221 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45378486-fd12-4a10-85f8-9b396079ba3e None admin] PATH_INFO: `/v3/services` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:34.548639 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45378486-fd12-4a10-85f8-9b396079ba3e None admin] RBAC: Adding query filter params (type=compute) {{(pid=80364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:50:34.548639 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45378486-fd12-4a10-85f8-9b396079ba3e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:34.549738 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45378486-fd12-4a10-85f8-9b396079ba3e None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:34.556151 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 502/1012] 10.222.0.27 () {56 vars in 1112 bytes} [Fri Aug 5 19:50:34 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:50:34.564521 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec716fa-b343-418e-8d3a-537d4d8707bc None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:34.673699 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec716fa-b343-418e-8d3a-537d4d8707bc None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:34.710777 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eccdeda-e9fa-4d63-9dd7-5e8d6d6b74d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:34.711396 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eccdeda-e9fa-4d63-9dd7-5e8d6d6b74d7 None admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:34.711670 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eccdeda-e9fa-4d63-9dd7-5e8d6d6b74d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:34.711935 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eccdeda-e9fa-4d63-9dd7-5e8d6d6b74d7 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:34.712689 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eccdeda-e9fa-4d63-9dd7-5e8d6d6b74d7 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:34.714278 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eccdeda-e9fa-4d63-9dd7-5e8d6d6b74d7 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:34.740529 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 511/1013] 10.222.0.27 () {60 vars in 1144 bytes} [Fri Aug 5 19:50:34 2022] POST /identity/v3/endpoints => generated 334 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 19:50:35.908613 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021659eb-ac26-4d48-90fb-424365354adf None None] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:35.909123 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021659eb-ac26-4d48-90fb-424365354adf None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:35.909234 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021659eb-ac26-4d48-90fb-424365354adf None None] PATH_INFO: `/` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:35.910516 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 503/1014] 10.222.0.27 () {54 vars in 927 bytes} [Fri Aug 5 19:50:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:50:35.915897 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f79986-3af5-411c-9c79-d27ab15e01d3 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:35.916304 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f79986-3af5-411c-9c79-d27ab15e01d3 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:35.916484 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f79986-3af5-411c-9c79-d27ab15e01d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:35.935141 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-59f79986-3af5-411c-9c79-d27ab15e01d3 None None] MFA Rules not processed for user `03d1834a64cd4a43829827cb99a72747`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:50:35.936342 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-59f79986-3af5-411c-9c79-d27ab15e01d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:50:35.974148 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:36.032756 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 512/1015] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Aug 5 19:50:35 2022] POST /identity/v3/auth/tokens => generated 1873 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 05 19:50:36.055030 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021659eb-ac26-4d48-90fb-424365354adf None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:36.183423 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021659eb-ac26-4d48-90fb-424365354adf None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:36.219243 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e476d9da-75b8-4eb0-aede-4dbae22dbc45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:36.219896 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e476d9da-75b8-4eb0-aede-4dbae22dbc45 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:36.220176 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e476d9da-75b8-4eb0-aede-4dbae22dbc45 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:36.220446 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e476d9da-75b8-4eb0-aede-4dbae22dbc45 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:36.221707 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e476d9da-75b8-4eb0-aede-4dbae22dbc45 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:36.223222 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e476d9da-75b8-4eb0-aede-4dbae22dbc45 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:36.225016 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 504/1016] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Aug 5 19:50:36 2022] GET /identity/v3/domains/default => generated 199 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:50:36.231681 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f79986-3af5-411c-9c79-d27ab15e01d3 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:36.341651 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f79986-3af5-411c-9c79-d27ab15e01d3 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:36.394332 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac49386d-e4dc-4e20-86ac-675a64f79ab0 None None] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:36.394570 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68479474-d643-4e1e-8847-12c1c9a14014 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:36.394845 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac49386d-e4dc-4e20-86ac-675a64f79ab0 None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:36.395071 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68479474-d643-4e1e-8847-12c1c9a14014 None admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:36.395183 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac49386d-e4dc-4e20-86ac-675a64f79ab0 None None] PATH_INFO: `/` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:36.395347 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68479474-d643-4e1e-8847-12c1c9a14014 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:36.395612 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68479474-d643-4e1e-8847-12c1c9a14014 None admin] PATH_INFO: `/v3/projects` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:36.396316 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68479474-d643-4e1e-8847-12c1c9a14014 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:36.397557 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 505/1017] 10.222.0.27 () {54 vars in 927 bytes} [Fri Aug 5 19:50:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:50:36.397793 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68479474-d643-4e1e-8847-12c1c9a14014 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:36.403023 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cef2b7-af18-4bda-b0f1-61aa1bfb2d80 None None] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:36.403360 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cef2b7-af18-4bda-b0f1-61aa1bfb2d80 None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:36.403635 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cef2b7-af18-4bda-b0f1-61aa1bfb2d80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:36.423661 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.auth.core [None req-83cef2b7-af18-4bda-b0f1-61aa1bfb2d80 None None] MFA Rules not processed for user `03d1834a64cd4a43829827cb99a72747`. Rule list: `[]` (Enabled: `True`). {{(pid=80364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:50:36.424979 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-83cef2b7-af18-4bda-b0f1-61aa1bfb2d80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:50:36.432096 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 513/1018] 10.222.0.27 () {60 vars in 1141 bytes} [Fri Aug 5 19:50:36 2022] POST /identity/v3/projects => generated 311 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 19:50:36.434902 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:36.520389 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 506/1019] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Aug 5 19:50:36 2022] POST /identity/v3/auth/tokens => generated 1873 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 05 19:50:36.541730 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68479474-d643-4e1e-8847-12c1c9a14014 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:36.562547 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:36.678605 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68479474-d643-4e1e-8847-12c1c9a14014 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:36.714102 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02cbc5d0-b786-4d17-bf79-b182c6877d53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:36.714677 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cbc5d0-b786-4d17-bf79-b182c6877d53 None admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:36.714966 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cbc5d0-b786-4d17-bf79-b182c6877d53 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:36.715240 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cbc5d0-b786-4d17-bf79-b182c6877d53 None admin] PATH_INFO: `/v3/services` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:36.715954 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02cbc5d0-b786-4d17-bf79-b182c6877d53 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:36.717359 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02cbc5d0-b786-4d17-bf79-b182c6877d53 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:36.733722 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 514/1020] 10.222.0.27 () {60 vars in 1141 bytes} [Fri Aug 5 19:50:36 2022] POST /identity/v3/services => generated 242 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 19:50:36.878391 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5688f15-dfc8-4259-ab32-665faf343fba None None] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:36.878741 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5688f15-dfc8-4259-ab32-665faf343fba None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:36.878965 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5688f15-dfc8-4259-ab32-665faf343fba None None] PATH_INFO: `/` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:36.880477 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 507/1021] 10.222.0.27 () {54 vars in 927 bytes} [Fri Aug 5 19:50:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:50:36.885779 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d15e89-0445-4528-8bda-f612dfb08ce6 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:36.885989 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d15e89-0445-4528-8bda-f612dfb08ce6 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:36.886239 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d15e89-0445-4528-8bda-f612dfb08ce6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:36.905820 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-a0d15e89-0445-4528-8bda-f612dfb08ce6 None None] MFA Rules not processed for user `03d1834a64cd4a43829827cb99a72747`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:50:36.907029 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-a0d15e89-0445-4528-8bda-f612dfb08ce6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:50:36.987794 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 515/1022] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Aug 5 19:50:36 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 05 19:50:37.014908 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5688f15-dfc8-4259-ab32-665faf343fba None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:37.123205 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5688f15-dfc8-4259-ab32-665faf343fba None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:37.158120 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6e133d-7bd3-441e-812f-e56f8e703e50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:37.158757 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6e133d-7bd3-441e-812f-e56f8e703e50 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:37.159035 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6e133d-7bd3-441e-812f-e56f8e703e50 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:37.159303 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6e133d-7bd3-441e-812f-e56f8e703e50 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:37.170196 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6e133d-7bd3-441e-812f-e56f8e703e50 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 19:50:37.175060 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application [None req-df6e133d-7bd3-441e-812f-e56f8e703e50 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 05 19:50:37.175060 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:50:37.175060 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:50:37.175060 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:50:37.175060 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:50:37.175060 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:50:37.175060 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:50:37.175060 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:50:37.175060 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:50:37.175060 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:50:37.175060 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:50:37.175060 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:50:37.175060 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 19:50:37.175060 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 19:50:37.175060 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 19:50:37.175060 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 19:50:37.176014 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 19:50:37.176014 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 19:50:37.176014 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 19:50:37.176014 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 19:50:37.176014 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:50:37.176014 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:50:37.176014 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:50:37.176014 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:50:37.176014 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:50:37.176014 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:50:37.176014 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:50:37.176014 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application with Lock( Aug 05 19:50:37.176014 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:50:37.176014 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:50:37.176014 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:50:37.176014 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:50:37.177220 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:50:37.177220 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:50:37.177220 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:50:37.177220 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:50:37.177220 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 19:50:37.177220 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 19:50:37.177220 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 19:50:37.177220 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 19:50:37.177220 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 19:50:37.177220 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 19:50:37.177220 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 05 19:50:37.177220 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Aug 05 19:50:37.180584 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 508/1023] 10.222.0.27 () {56 vars in 1120 bytes} [Fri Aug 5 19:50:37 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:50:37.182699 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d15e89-0445-4528-8bda-f612dfb08ce6 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:37.184970 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:37.286574 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d15e89-0445-4528-8bda-f612dfb08ce6 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:37.320895 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4505026-3199-441a-8845-a3c66f25d9a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:37.321522 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4505026-3199-441a-8845-a3c66f25d9a5 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:37.321805 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4505026-3199-441a-8845-a3c66f25d9a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:37.322083 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4505026-3199-441a-8845-a3c66f25d9a5 None admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:37.322803 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4505026-3199-441a-8845-a3c66f25d9a5 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:50:37.323113 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4505026-3199-441a-8845-a3c66f25d9a5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:37.325048 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4505026-3199-441a-8845-a3c66f25d9a5 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:37.337974 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 516/1024] 10.222.0.27 () {56 vars in 1115 bytes} [Fri Aug 5 19:50:37 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:50:37.339246 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6e133d-7bd3-441e-812f-e56f8e703e50 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:37.343363 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:37.451602 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6e133d-7bd3-441e-812f-e56f8e703e50 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:37.485785 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a412a317-d130-4a3f-9f05-e329445fa3be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:37.486428 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a412a317-d130-4a3f-9f05-e329445fa3be None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:37.486708 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a412a317-d130-4a3f-9f05-e329445fa3be None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:37.486980 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a412a317-d130-4a3f-9f05-e329445fa3be None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:37.497654 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a412a317-d130-4a3f-9f05-e329445fa3be None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:37.499094 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a412a317-d130-4a3f-9f05-e329445fa3be None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:37.514516 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application [None req-a412a317-d130-4a3f-9f05-e329445fa3be None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:50:37.514516 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:50:37.514516 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:50:37.514516 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:50:37.514516 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:50:37.514516 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:50:37.514516 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:50:37.514516 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:50:37.514516 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:50:37.514516 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:50:37.514516 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:50:37.514516 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:50:37.514516 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 19:50:37.514516 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 19:50:37.514516 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 19:50:37.515429 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 19:50:37.515429 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:50:37.515429 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:50:37.515429 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:50:37.515429 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:50:37.515429 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:50:37.515429 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:50:37.515429 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:50:37.515429 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application with Lock( Aug 05 19:50:37.515429 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:50:37.515429 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:50:37.515429 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:50:37.515429 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:50:37.515429 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:50:37.515429 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:50:37.515429 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:50:37.515429 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:50:37.516319 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 19:50:37.516319 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 19:50:37.516319 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:50:37.516319 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Aug 05 19:50:37.520486 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 509/1025] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Aug 5 19:50:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:50:37.521595 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4505026-3199-441a-8845-a3c66f25d9a5 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:37.592786 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:37.635992 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4505026-3199-441a-8845-a3c66f25d9a5 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:37.672475 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e166872c-950e-4a5d-be98-eed59f6f99ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:37.673130 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e166872c-950e-4a5d-be98-eed59f6f99ab None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:37.673409 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e166872c-950e-4a5d-be98-eed59f6f99ab None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:37.673680 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e166872c-950e-4a5d-be98-eed59f6f99ab None admin] PATH_INFO: `/v3/domains` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:37.674382 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e166872c-950e-4a5d-be98-eed59f6f99ab None admin] RBAC: Adding query filter params (name=Default) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:50:37.674687 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e166872c-950e-4a5d-be98-eed59f6f99ab None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:37.676052 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e166872c-950e-4a5d-be98-eed59f6f99ab None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:37.693646 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 517/1026] 10.222.0.27 () {56 vars in 1109 bytes} [Fri Aug 5 19:50:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:50:37.695235 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a412a317-d130-4a3f-9f05-e329445fa3be None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:37.760096 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:37.811498 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a412a317-d130-4a3f-9f05-e329445fa3be None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:37.848861 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bed394c-72ba-42c1-8dd5-73cb0c996e51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:37.849404 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bed394c-72ba-42c1-8dd5-73cb0c996e51 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:37.849675 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bed394c-72ba-42c1-8dd5-73cb0c996e51 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:37.849949 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bed394c-72ba-42c1-8dd5-73cb0c996e51 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:37.858339 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bed394c-72ba-42c1-8dd5-73cb0c996e51 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:37.859768 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bed394c-72ba-42c1-8dd5-73cb0c996e51 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:37.873333 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application [None req-1bed394c-72ba-42c1-8dd5-73cb0c996e51 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 05 19:50:37.873333 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:50:37.873333 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:50:37.873333 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:50:37.873333 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:50:37.873333 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:50:37.873333 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:50:37.873333 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:50:37.873333 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:50:37.873333 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:50:37.873333 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:50:37.873333 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:50:37.873333 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 05 19:50:37.873333 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 05 19:50:37.873333 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 05 19:50:37.873333 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 05 19:50:37.874538 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:50:37.874538 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:50:37.874538 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 05 19:50:37.874538 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 19:50:37.874538 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 05 19:50:37.874538 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 19:50:37.874538 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:50:37.874538 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:50:37.874538 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:50:37.874538 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:50:37.874538 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:50:37.874538 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application with Lock( Aug 05 19:50:37.874538 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:50:37.874538 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:50:37.874538 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:50:37.874538 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:50:37.874538 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:50:37.874538 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:50:37.875687 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:50:37.875687 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:50:37.875687 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 05 19:50:37.875687 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 05 19:50:37.875687 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 05 19:50:37.875687 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 05 19:50:37.875687 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 05 19:50:37.875687 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 05 19:50:37.875687 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 05 19:50:37.875687 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Aug 05 19:50:37.875687 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 510/1027] 10.222.0.27 () {56 vars in 1093 bytes} [Fri Aug 5 19:50:37 2022] GET /identity/v3/users/nova => generated 82 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:50:37.880777 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e166872c-950e-4a5d-be98-eed59f6f99ab None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:38.007841 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e166872c-950e-4a5d-be98-eed59f6f99ab None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:38.032077 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:38.044918 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647402f7-3845-4fed-9cf3-5b5d25c3ef30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:38.045536 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647402f7-3845-4fed-9cf3-5b5d25c3ef30 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:38.045811 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647402f7-3845-4fed-9cf3-5b5d25c3ef30 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:38.046078 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647402f7-3845-4fed-9cf3-5b5d25c3ef30 None admin] PATH_INFO: `/v3/users` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:38.046867 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-647402f7-3845-4fed-9cf3-5b5d25c3ef30 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:50:38.047167 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-647402f7-3845-4fed-9cf3-5b5d25c3ef30 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:38.048607 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-647402f7-3845-4fed-9cf3-5b5d25c3ef30 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:38.071126 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 518/1028] 10.222.0.27 () {56 vars in 1133 bytes} [Fri Aug 5 19:50:37 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 19:50:38.072814 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bed394c-72ba-42c1-8dd5-73cb0c996e51 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:38.178539 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bed394c-72ba-42c1-8dd5-73cb0c996e51 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:38.192760 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:38.213819 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c55192-bb7b-4f70-8755-db3168ed0b9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:38.214440 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c55192-bb7b-4f70-8755-db3168ed0b9f None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:38.215240 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c55192-bb7b-4f70-8755-db3168ed0b9f None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:38.215514 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c55192-bb7b-4f70-8755-db3168ed0b9f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:38.286266 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c55192-bb7b-4f70-8755-db3168ed0b9f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:38.288871 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c55192-bb7b-4f70-8755-db3168ed0b9f None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:38.397770 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 511/1029] 10.222.0.27 () {58 vars in 1282 bytes} [Fri Aug 5 19:50:38 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 05 19:50:38.403767 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647402f7-3845-4fed-9cf3-5b5d25c3ef30 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:38.514207 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647402f7-3845-4fed-9cf3-5b5d25c3ef30 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:38.552336 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a598eeaf-e94b-486f-846f-5d9857793906 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:38.553037 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a598eeaf-e94b-486f-846f-5d9857793906 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:38.553318 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a598eeaf-e94b-486f-846f-5d9857793906 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:38.553524 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb74c36-348e-4f93-8631-1995c5d851d7 None None] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:38.553648 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a598eeaf-e94b-486f-846f-5d9857793906 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:38.553818 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb74c36-348e-4f93-8631-1995c5d851d7 None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:38.554208 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb74c36-348e-4f93-8631-1995c5d851d7 None None] PATH_INFO: `/` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:38.555840 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 512/1030] 10.222.0.27 () {54 vars in 927 bytes} [Fri Aug 5 19:50:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 05 19:50:38.561661 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7452a5c-a80b-45ac-85b4-6e42f01bf065 None None] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:38.561960 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7452a5c-a80b-45ac-85b4-6e42f01bf065 None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:38.562238 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7452a5c-a80b-45ac-85b4-6e42f01bf065 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:38.566108 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a598eeaf-e94b-486f-846f-5d9857793906 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:38.567764 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a598eeaf-e94b-486f-846f-5d9857793906 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:38.580541 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.auth.core [None req-d7452a5c-a80b-45ac-85b4-6e42f01bf065 None None] MFA Rules not processed for user `03d1834a64cd4a43829827cb99a72747`. Rule list: `[]` (Enabled: `True`). {{(pid=80364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:50:38.581745 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-d7452a5c-a80b-45ac-85b4-6e42f01bf065 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:50:38.584155 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-a598eeaf-e94b-486f-846f-5d9857793906 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:50:38.584155 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:50:38.584155 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:50:38.584155 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:50:38.584155 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:50:38.584155 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:50:38.584155 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:50:38.584155 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:50:38.584155 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:50:38.584155 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:50:38.584155 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:50:38.584155 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:50:38.584155 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 19:50:38.584155 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 19:50:38.584155 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 19:50:38.585198 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 19:50:38.585198 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:50:38.585198 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:50:38.585198 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:50:38.585198 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:50:38.585198 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:50:38.585198 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:50:38.585198 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:50:38.585198 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Aug 05 19:50:38.585198 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:50:38.585198 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:50:38.585198 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:50:38.585198 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:50:38.585198 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:50:38.585198 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:50:38.585198 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:50:38.585198 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:50:38.586305 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 19:50:38.586305 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 19:50:38.586305 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:50:38.586305 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Aug 05 19:50:38.592589 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 519/1031] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Aug 5 19:50:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:50:38.593428 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:38.593564 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a598eeaf-e94b-486f-846f-5d9857793906 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:38.630039 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:38.657292 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 513/1032] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Aug 5 19:50:38 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 05 19:50:38.682867 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7452a5c-a80b-45ac-85b4-6e42f01bf065 None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:38.705162 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a598eeaf-e94b-486f-846f-5d9857793906 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:38.743268 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e781b1-2661-4ee9-af2d-62ed9bcfbd9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:38.744423 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e781b1-2661-4ee9-af2d-62ed9bcfbd9a None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:38.744736 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e781b1-2661-4ee9-af2d-62ed9bcfbd9a None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:38.745012 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e781b1-2661-4ee9-af2d-62ed9bcfbd9a None admin] PATH_INFO: `/v3/domains` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:38.745711 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e781b1-2661-4ee9-af2d-62ed9bcfbd9a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:50:38.746581 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e781b1-2661-4ee9-af2d-62ed9bcfbd9a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:38.747957 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e781b1-2661-4ee9-af2d-62ed9bcfbd9a None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:38.766135 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 520/1033] 10.222.0.27 () {56 vars in 1109 bytes} [Fri Aug 5 19:50:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 19:50:38.766860 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e781b1-2661-4ee9-af2d-62ed9bcfbd9a None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:38.783213 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7452a5c-a80b-45ac-85b4-6e42f01bf065 None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:38.818972 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed5f755-777f-4220-a0c2-1bd30b8613f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:38.819713 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed5f755-777f-4220-a0c2-1bd30b8613f7 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:38.820063 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed5f755-777f-4220-a0c2-1bd30b8613f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:38.820408 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed5f755-777f-4220-a0c2-1bd30b8613f7 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:38.821663 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed5f755-777f-4220-a0c2-1bd30b8613f7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:38.823179 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed5f755-777f-4220-a0c2-1bd30b8613f7 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:38.825116 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 514/1034] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Aug 5 19:50:38 2022] GET /identity/v3/domains/default => generated 199 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 19:50:38.830701 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed5f755-777f-4220-a0c2-1bd30b8613f7 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:38.845092 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:38.876076 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:38.883178 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e781b1-2661-4ee9-af2d-62ed9bcfbd9a None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:38.920461 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a53d1e-8e41-46ad-b62d-535fa508973e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:38.921151 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a53d1e-8e41-46ad-b62d-535fa508973e None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:38.921433 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a53d1e-8e41-46ad-b62d-535fa508973e None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:38.921707 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a53d1e-8e41-46ad-b62d-535fa508973e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:38.928909 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a53d1e-8e41-46ad-b62d-535fa508973e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:38.930358 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a53d1e-8e41-46ad-b62d-535fa508973e None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:38.942395 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-98a53d1e-8e41-46ad-b62d-535fa508973e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 19:50:38.942395 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:50:38.942395 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:50:38.942395 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:50:38.942395 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:50:38.942395 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:50:38.942395 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:50:38.942395 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:50:38.942395 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:50:38.942395 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:50:38.942395 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:50:38.942395 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:50:38.942395 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 05 19:50:38.942395 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 05 19:50:38.942395 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 05 19:50:38.944225 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 05 19:50:38.944225 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:50:38.944225 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:50:38.944225 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:50:38.944225 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:50:38.944225 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:50:38.944225 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:50:38.944225 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:50:38.944225 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Aug 05 19:50:38.944225 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:50:38.944225 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:50:38.944225 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:50:38.944225 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:50:38.944225 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:50:38.944225 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:50:38.944225 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:50:38.944225 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:50:38.945349 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 05 19:50:38.945349 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 05 19:50:38.945349 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 05 19:50:38.945349 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 05 19:50:38.945349 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 05 19:50:38.945349 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 05 19:50:38.945349 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 19:50:38.945349 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Aug 05 19:50:38.945349 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 521/1035] 10.222.0.27 () {56 vars in 1111 bytes} [Fri Aug 5 19:50:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 05 19:50:38.945349 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d3e422-4e01-41fb-a6ea-6f6d35452aad None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:38.945349 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d3e422-4e01-41fb-a6ea-6f6d35452aad None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:38.945946 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d3e422-4e01-41fb-a6ea-6f6d35452aad None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:38.948670 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 522/1036] 10.222.0.27 () {54 vars in 927 bytes} [Fri Aug 5 19:50:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:50:38.953124 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d3e422-4e01-41fb-a6ea-6f6d35452aad None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:38.958101 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed5f755-777f-4220-a0c2-1bd30b8613f7 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:38.991414 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa07d8e-9515-4c82-8e68-2b3ed63981a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:38.992023 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa07d8e-9515-4c82-8e68-2b3ed63981a3 None admin] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:38.992299 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa07d8e-9515-4c82-8e68-2b3ed63981a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:38.992596 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa07d8e-9515-4c82-8e68-2b3ed63981a3 None admin] PATH_INFO: `/v3/users` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:38.993270 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa07d8e-9515-4c82-8e68-2b3ed63981a3 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:38.994694 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa07d8e-9515-4c82-8e68-2b3ed63981a3 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:39.018109 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 515/1037] 10.222.0.27 () {60 vars in 1132 bytes} [Fri Aug 5 19:50:38 2022] POST /identity/v3/users => generated 285 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 05 19:50:39.022689 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2183cc-aa8b-4cfc-99f6-c221c9d0fa74 None None] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:39.022980 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2183cc-aa8b-4cfc-99f6-c221c9d0fa74 None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:39.023499 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2183cc-aa8b-4cfc-99f6-c221c9d0fa74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:39.043558 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.auth.core [None req-4c2183cc-aa8b-4cfc-99f6-c221c9d0fa74 None None] MFA Rules not processed for user `03d1834a64cd4a43829827cb99a72747`. Rule list: `[]` (Enabled: `True`). {{(pid=80364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:50:39.044839 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-4c2183cc-aa8b-4cfc-99f6-c221c9d0fa74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:50:39.062250 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d3e422-4e01-41fb-a6ea-6f6d35452aad None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:39.103375 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dbfdc8e-d215-46b6-8464-aac88bf8df37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:39.104033 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbfdc8e-d215-46b6-8464-aac88bf8df37 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:39.104288 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbfdc8e-d215-46b6-8464-aac88bf8df37 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:39.104590 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbfdc8e-d215-46b6-8464-aac88bf8df37 None admin] PATH_INFO: `/v3/projects` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:39.105404 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dbfdc8e-d215-46b6-8464-aac88bf8df37 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:50:39.105714 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dbfdc8e-d215-46b6-8464-aac88bf8df37 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:39.107181 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dbfdc8e-d215-46b6-8464-aac88bf8df37 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:39.118026 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:39.120216 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 523/1038] 10.222.0.27 () {56 vars in 1148 bytes} [Fri Aug 5 19:50:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:50:39.127493 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dbfdc8e-d215-46b6-8464-aac88bf8df37 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:39.128973 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 516/1039] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Aug 5 19:50:39 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 05 19:50:39.145935 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:39.154772 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2183cc-aa8b-4cfc-99f6-c221c9d0fa74 None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:39.246944 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dbfdc8e-d215-46b6-8464-aac88bf8df37 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:39.274777 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2183cc-aa8b-4cfc-99f6-c221c9d0fa74 None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:39.283337 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac9a7df-edfd-4ea5-bc9f-7f7c9fd31372 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:39.284281 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac9a7df-edfd-4ea5-bc9f-7f7c9fd31372 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:39.284799 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac9a7df-edfd-4ea5-bc9f-7f7c9fd31372 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:39.285181 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac9a7df-edfd-4ea5-bc9f-7f7c9fd31372 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:39.286067 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac9a7df-edfd-4ea5-bc9f-7f7c9fd31372 None admin] RBAC: Adding query filter params (role.id=31647643dd7c43aba270f63d5fe83acb, scope.project.id=a70a82dcfc574fad90b3f5f00abbba58, user.id=11001a2ad96f4ec8a35aab2c64376875) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:50:39.286453 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac9a7df-edfd-4ea5-bc9f-7f7c9fd31372 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:39.287985 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac9a7df-edfd-4ea5-bc9f-7f7c9fd31372 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:39.294389 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 524/1040] 10.222.0.27 () {56 vars in 1374 bytes} [Fri Aug 5 19:50:39 2022] GET /identity/v3/role_assignments?user.id=11001a2ad96f4ec8a35aab2c64376875&scope.project.id=a70a82dcfc574fad90b3f5f00abbba58&role.id=31647643dd7c43aba270f63d5fe83acb => generated 259 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:50:39.318303 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519b8825-0e4a-4472-bd1a-7b33961dab70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:39.318992 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519b8825-0e4a-4472-bd1a-7b33961dab70 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:39.319276 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519b8825-0e4a-4472-bd1a-7b33961dab70 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:39.319553 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519b8825-0e4a-4472-bd1a-7b33961dab70 None admin] PATH_INFO: `/v3/services/image` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:39.320231 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519b8825-0e4a-4472-bd1a-7b33961dab70 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:39.322062 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519b8825-0e4a-4472-bd1a-7b33961dab70 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:39.332185 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application [None req-519b8825-0e4a-4472-bd1a-7b33961dab70 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 05 19:50:39.332185 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:50:39.332185 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 05 19:50:39.332185 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 05 19:50:39.332185 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 05 19:50:39.332185 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 05 19:50:39.332185 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 05 19:50:39.332185 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 05 19:50:39.332185 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 05 19:50:39.332185 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Aug 05 19:50:39.332185 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 05 19:50:39.332185 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Aug 05 19:50:39.332185 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:50:39.332185 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:50:39.332185 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:50:39.332185 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:50:39.332185 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:50:39.333552 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:50:39.333552 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:50:39.333552 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:50:39.333552 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:50:39.333552 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:50:39.333552 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:50:39.333552 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 05 19:50:39.333552 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 05 19:50:39.333552 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 05 19:50:39.333552 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 05 19:50:39.333552 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:50:39.333552 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:50:39.333552 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:50:39.333552 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:50:39.333552 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:50:39.333552 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:50:39.334420 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:50:39.334420 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application with Lock( Aug 05 19:50:39.334420 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:50:39.334420 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:50:39.334420 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:50:39.334420 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:50:39.334420 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:50:39.334420 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:50:39.334420 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:50:39.334420 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:50:39.334420 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 05 19:50:39.334420 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 05 19:50:39.334420 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 05 19:50:39.334420 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Aug 05 19:50:39.334420 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 517/1041] 10.222.0.27 () {56 vars in 1105 bytes} [Fri Aug 5 19:50:39 2022] GET /identity/v3/services/image => generated 86 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 05 19:50:39.339755 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac9a7df-edfd-4ea5-bc9f-7f7c9fd31372 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:39.442268 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:39.467662 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac9a7df-edfd-4ea5-bc9f-7f7c9fd31372 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:39.502287 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f420c2-b124-407c-a4d9-d7c9ed41abdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:39.502915 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f420c2-b124-407c-a4d9-d7c9ed41abdd None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:39.503200 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f420c2-b124-407c-a4d9-d7c9ed41abdd None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:39.503474 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f420c2-b124-407c-a4d9-d7c9ed41abdd None admin] PATH_INFO: `/v3/services` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:39.504188 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f420c2-b124-407c-a4d9-d7c9ed41abdd None admin] RBAC: Adding query filter params (name=image) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:50:39.504516 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f420c2-b124-407c-a4d9-d7c9ed41abdd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:39.505891 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f420c2-b124-407c-a4d9-d7c9ed41abdd None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:39.516107 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 525/1042] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Aug 5 19:50:39 2022] GET /identity/v3/services?name=image => generated 122 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:50:39.517576 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519b8825-0e4a-4472-bd1a-7b33961dab70 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:39.583532 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:39.629571 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519b8825-0e4a-4472-bd1a-7b33961dab70 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:39.673750 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cba5134-4781-473d-a3fa-d8b6bcca42a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:39.674385 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cba5134-4781-473d-a3fa-d8b6bcca42a4 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:39.674669 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cba5134-4781-473d-a3fa-d8b6bcca42a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:39.674946 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cba5134-4781-473d-a3fa-d8b6bcca42a4 None admin] PATH_INFO: `/v3/services` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:39.675652 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cba5134-4781-473d-a3fa-d8b6bcca42a4 None admin] RBAC: Adding query filter params (type=image) {{(pid=80364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:50:39.675946 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cba5134-4781-473d-a3fa-d8b6bcca42a4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:39.677418 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cba5134-4781-473d-a3fa-d8b6bcca42a4 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:39.684822 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 518/1043] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Aug 5 19:50:39 2022] GET /identity/v3/services?type=image => generated 350 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 19:50:39.690607 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f420c2-b124-407c-a4d9-d7c9ed41abdd None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:39.856618 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f420c2-b124-407c-a4d9-d7c9ed41abdd None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:39.909979 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1687c42f-0cd8-43a1-af83-c921ec15d6a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:39.910855 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1687c42f-0cd8-43a1-af83-c921ec15d6a2 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:39.911309 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1687c42f-0cd8-43a1-af83-c921ec15d6a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:39.911760 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1687c42f-0cd8-43a1-af83-c921ec15d6a2 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:39.913045 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1687c42f-0cd8-43a1-af83-c921ec15d6a2 None admin] RBAC: Adding query filter params (interface=public, service_id=90fb7d2b4c824b3590ebdf852f2b53da, region_id=RegionOne) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:50:39.913564 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1687c42f-0cd8-43a1-af83-c921ec15d6a2 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:39.915943 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1687c42f-0cd8-43a1-af83-c921ec15d6a2 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:39.932566 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 526/1044] 10.222.0.27 () {56 vars in 1251 bytes} [Fri Aug 5 19:50:39 2022] GET /identity/v3/endpoints?service_id=90fb7d2b4c824b3590ebdf852f2b53da&interface=public®ion_id=RegionOne => generated 194 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:50:39.934887 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cba5134-4781-473d-a3fa-d8b6bcca42a4 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:40.046349 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cba5134-4781-473d-a3fa-d8b6bcca42a4 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:40.081756 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:40.091098 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0daabb0b-4a60-4af4-9593-87b02aafc6ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:40.091721 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0daabb0b-4a60-4af4-9593-87b02aafc6ee None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:40.092006 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0daabb0b-4a60-4af4-9593-87b02aafc6ee None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:40.092285 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0daabb0b-4a60-4af4-9593-87b02aafc6ee None admin] PATH_INFO: `/v3/services` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:40.092974 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0daabb0b-4a60-4af4-9593-87b02aafc6ee None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:40.094349 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0daabb0b-4a60-4af4-9593-87b02aafc6ee None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:40.101574 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 519/1045] 10.222.0.27 () {56 vars in 1087 bytes} [Fri Aug 5 19:50:39 2022] GET /identity/v3/services => generated 1957 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 05 19:50:41.185956 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab047c2-ea1c-4836-85c9-d078c9703820 None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:41.186552 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab047c2-ea1c-4836-85c9-d078c9703820 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:41.186552 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab047c2-ea1c-4836-85c9-d078c9703820 None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:41.187919 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 527/1046] 10.222.0.27 () {54 vars in 927 bytes} [Fri Aug 5 19:50:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:50:41.193151 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40605d2-d350-4d02-b620-a56162fa76be None None] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:41.193639 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40605d2-d350-4d02-b620-a56162fa76be None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:41.193721 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40605d2-d350-4d02-b620-a56162fa76be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:41.215116 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.auth.core [None req-d40605d2-d350-4d02-b620-a56162fa76be None None] MFA Rules not processed for user `03d1834a64cd4a43829827cb99a72747`. Rule list: `[]` (Enabled: `True`). {{(pid=80364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:50:41.216361 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-d40605d2-d350-4d02-b620-a56162fa76be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:50:41.288558 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 520/1047] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Aug 5 19:50:41 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 05 19:50:41.311424 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab047c2-ea1c-4836-85c9-d078c9703820 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:41.321076 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:41.333789 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5e2c35-f1ab-40c0-80b9-497a5966e244 None None] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:41.334075 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5e2c35-f1ab-40c0-80b9-497a5966e244 None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:41.334357 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5e2c35-f1ab-40c0-80b9-497a5966e244 None None] PATH_INFO: `/` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:41.335982 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 521/1048] 10.222.0.27 () {54 vars in 927 bytes} [Fri Aug 5 19:50:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:50:41.342363 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506cad28-0ebe-4d4b-9182-10132ecf81b5 None None] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:41.342673 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506cad28-0ebe-4d4b-9182-10132ecf81b5 None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:41.342926 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506cad28-0ebe-4d4b-9182-10132ecf81b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:41.367179 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.auth.core [None req-506cad28-0ebe-4d4b-9182-10132ecf81b5 None None] MFA Rules not processed for user `03d1834a64cd4a43829827cb99a72747`. Rule list: `[]` (Enabled: `True`). {{(pid=80364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:50:41.367179 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-506cad28-0ebe-4d4b-9182-10132ecf81b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:50:41.432736 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab047c2-ea1c-4836-85c9-d078c9703820 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:41.452625 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 522/1049] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Aug 5 19:50:41 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 05 19:50:41.469793 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f4c89c-4a84-4733-9471-5b2567e45741 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:41.470520 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f4c89c-4a84-4733-9471-5b2567e45741 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:41.470798 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f4c89c-4a84-4733-9471-5b2567e45741 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:41.471064 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f4c89c-4a84-4733-9471-5b2567e45741 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:41.471974 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506cad28-0ebe-4d4b-9182-10132ecf81b5 None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:41.482603 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f4c89c-4a84-4733-9471-5b2567e45741 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 19:50:41.487410 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-73f4c89c-4a84-4733-9471-5b2567e45741 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 19:50:41.487410 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:50:41.487410 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:50:41.487410 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:50:41.487410 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:50:41.487410 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:50:41.487410 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:50:41.487410 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:50:41.487410 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:50:41.487410 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:50:41.487410 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:50:41.487410 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:50:41.487410 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 19:50:41.487410 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 19:50:41.487410 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 19:50:41.487410 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 19:50:41.488408 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 19:50:41.488408 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 19:50:41.488408 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 19:50:41.488408 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 19:50:41.488408 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:50:41.488408 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:50:41.488408 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:50:41.488408 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:50:41.488408 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:50:41.488408 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:50:41.488408 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:50:41.488408 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Aug 05 19:50:41.488408 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:50:41.488408 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:50:41.488408 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:50:41.488408 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:50:41.489305 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:50:41.489305 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:50:41.489305 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:50:41.489305 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:50:41.489305 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 19:50:41.489305 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 19:50:41.489305 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 19:50:41.489305 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 19:50:41.489305 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 19:50:41.489305 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 19:50:41.489305 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 19:50:41.489305 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Aug 05 19:50:41.489305 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 528/1050] 10.222.0.27 () {56 vars in 1096 bytes} [Fri Aug 5 19:50:41 2022] GET /identity/v3/roles/admin => generated 83 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:50:41.495756 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f4c89c-4a84-4733-9471-5b2567e45741 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:41.530871 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:41.571835 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:41.589141 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506cad28-0ebe-4d4b-9182-10132ecf81b5 None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:41.610834 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f4c89c-4a84-4733-9471-5b2567e45741 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:41.630977 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d60741-c7be-4c79-847d-614839a10885 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:41.631999 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d60741-c7be-4c79-847d-614839a10885 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:41.632345 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d60741-c7be-4c79-847d-614839a10885 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:41.632681 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d60741-c7be-4c79-847d-614839a10885 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:41.644058 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d60741-c7be-4c79-847d-614839a10885 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 19:50:41.649411 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application [None req-50d60741-c7be-4c79-847d-614839a10885 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 05 19:50:41.649411 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:50:41.649411 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:50:41.649411 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:50:41.649411 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:50:41.649411 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:50:41.649411 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:50:41.649411 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:50:41.649411 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:50:41.649411 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:50:41.649411 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:50:41.649411 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:50:41.649411 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 19:50:41.649411 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 19:50:41.649411 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 19:50:41.649411 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 19:50:41.650410 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 19:50:41.650410 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 19:50:41.650410 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 19:50:41.650410 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 19:50:41.650410 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:50:41.650410 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:50:41.650410 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:50:41.650410 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:50:41.650410 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:50:41.650410 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:50:41.650410 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:50:41.650410 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application with Lock( Aug 05 19:50:41.650410 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:50:41.650410 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:50:41.650410 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:50:41.650410 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:50:41.651244 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:50:41.651244 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:50:41.651244 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:50:41.651244 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:50:41.651244 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 19:50:41.651244 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 19:50:41.651244 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 19:50:41.651244 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 19:50:41.651244 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 19:50:41.651244 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 19:50:41.651244 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 05 19:50:41.651244 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Aug 05 19:50:41.651244 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 523/1051] 10.222.0.27 () {56 vars in 1120 bytes} [Fri Aug 5 19:50:41 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 05 19:50:41.654036 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce2ac20-6b96-4d97-a119-f39098b5f253 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:41.654663 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce2ac20-6b96-4d97-a119-f39098b5f253 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:41.654944 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce2ac20-6b96-4d97-a119-f39098b5f253 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:41.655220 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce2ac20-6b96-4d97-a119-f39098b5f253 None admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:41.655947 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce2ac20-6b96-4d97-a119-f39098b5f253 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:50:41.656470 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce2ac20-6b96-4d97-a119-f39098b5f253 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:41.657315 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d60741-c7be-4c79-847d-614839a10885 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:41.657723 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce2ac20-6b96-4d97-a119-f39098b5f253 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:41.667413 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 529/1052] 10.222.0.27 () {56 vars in 1099 bytes} [Fri Aug 5 19:50:41 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:50:41.673082 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce2ac20-6b96-4d97-a119-f39098b5f253 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:41.772364 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d60741-c7be-4c79-847d-614839a10885 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:41.784411 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:41.796086 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce2ac20-6b96-4d97-a119-f39098b5f253 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:41.821886 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f00297-06b5-49ff-af2a-160f64a1fe40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:41.822674 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f00297-06b5-49ff-af2a-160f64a1fe40 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:41.822959 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f00297-06b5-49ff-af2a-160f64a1fe40 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:41.823234 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f00297-06b5-49ff-af2a-160f64a1fe40 None admin] PATH_INFO: `/v3/roles` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:41.825756 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f00297-06b5-49ff-af2a-160f64a1fe40 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:50:41.825756 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f00297-06b5-49ff-af2a-160f64a1fe40 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:41.826452 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f00297-06b5-49ff-af2a-160f64a1fe40 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:41.829530 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:41.842115 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 524/1053] 10.222.0.27 () {56 vars in 1115 bytes} [Fri Aug 5 19:50:41 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 19:50:41.844736 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ab72ac-e7ce-4a54-8e59-e35de35decf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:41.845804 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f00297-06b5-49ff-af2a-160f64a1fe40 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:41.845929 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ab72ac-e7ce-4a54-8e59-e35de35decf9 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:41.845929 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ab72ac-e7ce-4a54-8e59-e35de35decf9 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:41.845929 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ab72ac-e7ce-4a54-8e59-e35de35decf9 None admin] PATH_INFO: `/v3/users/fb6447a57a204dfca40b3f6ffd36b5cf` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:41.863081 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ab72ac-e7ce-4a54-8e59-e35de35decf9 None admin] RBAC: Authorizing `identity:get_user(user_id=fb6447a57a204dfca40b3f6ffd36b5cf)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:41.865392 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ab72ac-e7ce-4a54-8e59-e35de35decf9 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:41.868543 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 530/1054] 10.222.0.27 () {56 vars in 1177 bytes} [Fri Aug 5 19:50:41 2022] GET /identity/v3/users/fb6447a57a204dfca40b3f6ffd36b5cf => generated 285 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 19:50:41.876142 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ab72ac-e7ce-4a54-8e59-e35de35decf9 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:41.968355 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f00297-06b5-49ff-af2a-160f64a1fe40 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:42.020881 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ab72ac-e7ce-4a54-8e59-e35de35decf9 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:42.023709 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7fbf1e-cf4a-4daa-a2bd-6c9278773bf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:42.024570 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7fbf1e-cf4a-4daa-a2bd-6c9278773bf4 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:42.024866 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7fbf1e-cf4a-4daa-a2bd-6c9278773bf4 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:42.025167 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7fbf1e-cf4a-4daa-a2bd-6c9278773bf4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:42.037664 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d7fbf1e-cf4a-4daa-a2bd-6c9278773bf4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:42.039150 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d7fbf1e-cf4a-4daa-a2bd-6c9278773bf4 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:42.054946 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application [None req-8d7fbf1e-cf4a-4daa-a2bd-6c9278773bf4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:50:42.054946 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:50:42.054946 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:50:42.054946 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:50:42.054946 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:50:42.054946 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:50:42.054946 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:50:42.054946 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:50:42.054946 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:50:42.054946 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:50:42.054946 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:50:42.054946 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:50:42.054946 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 19:50:42.054946 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 19:50:42.054946 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 19:50:42.055926 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 19:50:42.055926 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:50:42.055926 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:50:42.055926 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:50:42.055926 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:50:42.055926 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:50:42.055926 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:50:42.055926 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:50:42.055926 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application with Lock( Aug 05 19:50:42.055926 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:50:42.055926 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:50:42.055926 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:50:42.055926 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:50:42.055926 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:50:42.055926 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:50:42.055926 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:50:42.055926 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:50:42.056863 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 19:50:42.056863 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 19:50:42.056863 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:50:42.056863 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Aug 05 19:50:42.056863 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 525/1055] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Aug 5 19:50:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 05 19:50:42.062462 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7fbf1e-cf4a-4daa-a2bd-6c9278773bf4 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:42.070617 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2882f3-4e8b-42e5-957b-c44d10ad39c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:42.071248 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2882f3-4e8b-42e5-957b-c44d10ad39c8 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:42.071534 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2882f3-4e8b-42e5-957b-c44d10ad39c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:42.072310 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2882f3-4e8b-42e5-957b-c44d10ad39c8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:42.089957 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:42.150867 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:42.167681 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb2882f3-4e8b-42e5-957b-c44d10ad39c8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:42.169838 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb2882f3-4e8b-42e5-957b-c44d10ad39c8 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:42.178871 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7fbf1e-cf4a-4daa-a2bd-6c9278773bf4 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:42.213665 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162fe08a-8be1-499f-a75c-a6efc92867b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:42.214330 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162fe08a-8be1-499f-a75c-a6efc92867b4 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:42.215126 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162fe08a-8be1-499f-a75c-a6efc92867b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:42.215431 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162fe08a-8be1-499f-a75c-a6efc92867b4 None admin] PATH_INFO: `/v3/domains` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:42.216156 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-162fe08a-8be1-499f-a75c-a6efc92867b4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:50:42.216466 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-162fe08a-8be1-499f-a75c-a6efc92867b4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:42.217917 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-162fe08a-8be1-499f-a75c-a6efc92867b4 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:42.232852 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 526/1056] 10.222.0.27 () {56 vars in 1109 bytes} [Fri Aug 5 19:50:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 19:50:42.232852 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3abbbb-988a-4359-9784-2e1e14680d03 None None] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:42.232852 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3abbbb-988a-4359-9784-2e1e14680d03 None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:42.233197 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3abbbb-988a-4359-9784-2e1e14680d03 None None] PATH_INFO: `/` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:42.234317 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 527/1057] 10.222.0.27 () {54 vars in 927 bytes} [Fri Aug 5 19:50:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:50:42.236678 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3abbbb-988a-4359-9784-2e1e14680d03 None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:42.284776 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 531/1058] 10.222.0.27 () {58 vars in 1282 bytes} [Fri Aug 5 19:50:41 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 409 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 05 19:50:42.285959 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e05dc7b-3b0b-4d5f-b68f-a5af34c2ec4d None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:42.286247 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e05dc7b-3b0b-4d5f-b68f-a5af34c2ec4d None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:42.286520 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e05dc7b-3b0b-4d5f-b68f-a5af34c2ec4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:42.304048 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-1e05dc7b-3b0b-4d5f-b68f-a5af34c2ec4d None None] MFA Rules not processed for user `03d1834a64cd4a43829827cb99a72747`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:50:42.305738 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-1e05dc7b-3b0b-4d5f-b68f-a5af34c2ec4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:50:42.329866 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:42.348753 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3abbbb-988a-4359-9784-2e1e14680d03 None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:42.377034 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 532/1059] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Aug 5 19:50:42 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 05 19:50:42.377887 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e05dc7b-3b0b-4d5f-b68f-a5af34c2ec4d None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:42.394609 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8192b49-069f-493a-9128-1bbea07f9125 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:42.395254 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8192b49-069f-493a-9128-1bbea07f9125 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:42.395631 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8192b49-069f-493a-9128-1bbea07f9125 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:42.395898 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8192b49-069f-493a-9128-1bbea07f9125 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:42.396330 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:42.408533 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8192b49-069f-493a-9128-1bbea07f9125 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:42.409996 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8192b49-069f-493a-9128-1bbea07f9125 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:42.422844 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application [None req-e8192b49-069f-493a-9128-1bbea07f9125 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 05 19:50:42.422844 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:50:42.422844 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:50:42.422844 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:50:42.422844 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:50:42.422844 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:50:42.422844 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:50:42.422844 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:50:42.422844 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:50:42.422844 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:50:42.422844 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:50:42.422844 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:50:42.422844 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 05 19:50:42.422844 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 05 19:50:42.422844 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 05 19:50:42.422844 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 05 19:50:42.423879 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:50:42.423879 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:50:42.423879 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 05 19:50:42.423879 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 19:50:42.423879 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 05 19:50:42.423879 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 19:50:42.423879 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:50:42.423879 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:50:42.423879 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:50:42.423879 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:50:42.423879 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:50:42.423879 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application with Lock( Aug 05 19:50:42.423879 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:50:42.423879 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:50:42.423879 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:50:42.423879 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:50:42.423879 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:50:42.423879 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:50:42.424949 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:50:42.424949 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:50:42.424949 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 05 19:50:42.424949 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 05 19:50:42.424949 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 05 19:50:42.424949 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 05 19:50:42.424949 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 05 19:50:42.424949 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 05 19:50:42.424949 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 05 19:50:42.424949 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Aug 05 19:50:42.424949 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 528/1060] 10.222.0.27 () {56 vars in 1093 bytes} [Fri Aug 5 19:50:42 2022] GET /identity/v3/users/nova => generated 82 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 05 19:50:42.424949 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8192b49-069f-493a-9128-1bbea07f9125 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:42.485041 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e05dc7b-3b0b-4d5f-b68f-a5af34c2ec4d None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:42.519022 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ba9061-a0ff-4072-bb6d-315fdb98c87f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:42.519668 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ba9061-a0ff-4072-bb6d-315fdb98c87f None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:42.519949 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ba9061-a0ff-4072-bb6d-315fdb98c87f None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:42.520222 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ba9061-a0ff-4072-bb6d-315fdb98c87f None admin] PATH_INFO: `/v3/projects/bf700dedec5842849da197ced9f0b0cb` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:42.521428 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ba9061-a0ff-4072-bb6d-315fdb98c87f None admin] RBAC: Authorizing `identity:get_project(project_id=bf700dedec5842849da197ced9f0b0cb)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:42.522849 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ba9061-a0ff-4072-bb6d-315fdb98c87f None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:42.524823 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 533/1061] 10.222.0.27 () {56 vars in 1186 bytes} [Fri Aug 5 19:50:42 2022] GET /identity/v3/projects/bf700dedec5842849da197ced9f0b0cb => generated 311 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:50:42.525422 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8192b49-069f-493a-9128-1bbea07f9125 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:42.525593 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ba9061-a0ff-4072-bb6d-315fdb98c87f None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:42.565452 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68186d76-4450-4158-b389-fb04041809a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:42.566012 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68186d76-4450-4158-b389-fb04041809a6 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:42.566294 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68186d76-4450-4158-b389-fb04041809a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:42.566572 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68186d76-4450-4158-b389-fb04041809a6 None admin] PATH_INFO: `/v3/services/image` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:42.567209 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68186d76-4450-4158-b389-fb04041809a6 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:42.568623 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68186d76-4450-4158-b389-fb04041809a6 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:42.578242 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application [None req-68186d76-4450-4158-b389-fb04041809a6 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 05 19:50:42.578242 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:50:42.578242 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 05 19:50:42.578242 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 05 19:50:42.578242 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 05 19:50:42.578242 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 05 19:50:42.578242 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 05 19:50:42.578242 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 05 19:50:42.578242 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 05 19:50:42.578242 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Aug 05 19:50:42.578242 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 05 19:50:42.578242 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Aug 05 19:50:42.578242 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:50:42.578242 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:50:42.578242 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:50:42.578242 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:50:42.578242 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:50:42.579493 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:50:42.579493 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:50:42.579493 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:50:42.579493 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:50:42.579493 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:50:42.579493 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:50:42.579493 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 05 19:50:42.579493 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 05 19:50:42.579493 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 05 19:50:42.579493 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 05 19:50:42.579493 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:50:42.579493 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:50:42.579493 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:50:42.579493 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:50:42.579493 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:50:42.579493 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:50:42.580583 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:50:42.580583 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application with Lock( Aug 05 19:50:42.580583 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:50:42.580583 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:50:42.580583 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:50:42.580583 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:50:42.580583 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:50:42.580583 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:50:42.580583 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:50:42.580583 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:50:42.580583 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 05 19:50:42.580583 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 05 19:50:42.580583 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 05 19:50:42.580583 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Aug 05 19:50:42.580583 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 529/1062] 10.222.0.27 () {56 vars in 1105 bytes} [Fri Aug 5 19:50:42 2022] GET /identity/v3/services/image => generated 86 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:50:42.581408 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68186d76-4450-4158-b389-fb04041809a6 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:42.592316 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:42.630154 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:42.653858 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ba9061-a0ff-4072-bb6d-315fdb98c87f None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:42.694095 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69b888b-f323-4d78-a52d-06de183c2fb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:42.694709 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69b888b-f323-4d78-a52d-06de183c2fb8 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:42.694986 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69b888b-f323-4d78-a52d-06de183c2fb8 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:42.695255 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69b888b-f323-4d78-a52d-06de183c2fb8 None admin] PATH_INFO: `/v3/users` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:42.695578 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68186d76-4450-4158-b389-fb04041809a6 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:42.696049 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e69b888b-f323-4d78-a52d-06de183c2fb8 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:50:42.696349 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e69b888b-f323-4d78-a52d-06de183c2fb8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:42.697810 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e69b888b-f323-4d78-a52d-06de183c2fb8 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:42.716184 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 534/1063] 10.222.0.27 () {56 vars in 1133 bytes} [Fri Aug 5 19:50:42 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:50:42.717062 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69b888b-f323-4d78-a52d-06de183c2fb8 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:42.746438 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7fb1c8a-064e-49de-acb3-04a9d600c7e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:42.747367 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fb1c8a-064e-49de-acb3-04a9d600c7e9 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:42.747791 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fb1c8a-064e-49de-acb3-04a9d600c7e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:42.748207 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fb1c8a-064e-49de-acb3-04a9d600c7e9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:42.749515 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7fb1c8a-064e-49de-acb3-04a9d600c7e9 None admin] RBAC: Adding query filter params (role.id=c173688a960b46e5be65f0ccd6fbd734, scope.project.id=bf700dedec5842849da197ced9f0b0cb, user.id=fb6447a57a204dfca40b3f6ffd36b5cf) {{(pid=80364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:50:42.749997 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7fb1c8a-064e-49de-acb3-04a9d600c7e9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:42.752309 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7fb1c8a-064e-49de-acb3-04a9d600c7e9 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:42.762106 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 530/1064] 10.222.0.27 () {56 vars in 1374 bytes} [Fri Aug 5 19:50:42 2022] GET /identity/v3/role_assignments?user.id=fb6447a57a204dfca40b3f6ffd36b5cf&scope.project.id=bf700dedec5842849da197ced9f0b0cb&role.id=c173688a960b46e5be65f0ccd6fbd734 => generated 259 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:50:42.762601 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7fb1c8a-064e-49de-acb3-04a9d600c7e9 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:42.857506 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69b888b-f323-4d78-a52d-06de183c2fb8 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:42.868034 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:42.906125 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7fb1c8a-064e-49de-acb3-04a9d600c7e9 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:42.906660 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76f5ab3-3e61-40d3-858c-bbfa4239a6ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:42.907311 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76f5ab3-3e61-40d3-858c-bbfa4239a6ef None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:42.907591 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76f5ab3-3e61-40d3-858c-bbfa4239a6ef None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:42.908124 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76f5ab3-3e61-40d3-858c-bbfa4239a6ef None admin] PATH_INFO: `/v3/services` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:42.909828 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d76f5ab3-3e61-40d3-858c-bbfa4239a6ef None admin] RBAC: Adding query filter params (name=image) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:50:42.910153 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d76f5ab3-3e61-40d3-858c-bbfa4239a6ef None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:42.911571 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d76f5ab3-3e61-40d3-858c-bbfa4239a6ef None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:42.919449 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 535/1065] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Aug 5 19:50:42 2022] GET /identity/v3/services?name=image => generated 122 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:50:42.925167 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76f5ab3-3e61-40d3-858c-bbfa4239a6ef None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:42.934781 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:42.949873 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eece895-019e-4603-a782-db01bbd0e85e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:42.951486 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eece895-019e-4603-a782-db01bbd0e85e None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:42.951969 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eece895-019e-4603-a782-db01bbd0e85e None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:42.952421 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eece895-019e-4603-a782-db01bbd0e85e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:42.965941 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eece895-019e-4603-a782-db01bbd0e85e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:42.967816 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eece895-019e-4603-a782-db01bbd0e85e None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:42.985621 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application [None req-2eece895-019e-4603-a782-db01bbd0e85e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:50:42.985621 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:50:42.985621 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:50:42.985621 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:50:42.985621 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:50:42.985621 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:50:42.985621 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:50:42.985621 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:50:42.985621 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:50:42.985621 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:50:42.985621 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:50:42.985621 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:50:42.985621 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 19:50:42.985621 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 19:50:42.985621 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 19:50:42.986646 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 19:50:42.986646 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:50:42.986646 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:50:42.986646 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:50:42.986646 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:50:42.986646 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:50:42.986646 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:50:42.986646 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:50:42.986646 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application with Lock( Aug 05 19:50:42.986646 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:50:42.986646 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:50:42.986646 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:50:42.986646 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:50:42.986646 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:50:42.986646 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:50:42.986646 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:50:42.986646 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:50:42.987577 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 19:50:42.987577 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 19:50:42.987577 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:50:42.987577 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Aug 05 19:50:42.987577 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 531/1066] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Aug 5 19:50:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:50:42.993723 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eece895-019e-4603-a782-db01bbd0e85e None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:43.041190 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76f5ab3-3e61-40d3-858c-bbfa4239a6ef None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:43.078533 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84623e04-e635-477a-aec5-37970678acf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:43.079182 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84623e04-e635-477a-aec5-37970678acf9 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:43.079460 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84623e04-e635-477a-aec5-37970678acf9 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:43.079734 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84623e04-e635-477a-aec5-37970678acf9 None admin] PATH_INFO: `/v3/services` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:43.080418 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84623e04-e635-477a-aec5-37970678acf9 None admin] RBAC: Adding query filter params (type=image) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:50:43.080748 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84623e04-e635-477a-aec5-37970678acf9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:43.082112 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84623e04-e635-477a-aec5-37970678acf9 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:43.087405 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 536/1067] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Aug 5 19:50:42 2022] GET /identity/v3/services?type=image => generated 350 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 19:50:43.093703 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84623e04-e635-477a-aec5-37970678acf9 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:43.135825 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eece895-019e-4603-a782-db01bbd0e85e None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:43.167463 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:43.182711 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab436ec-0cc1-47eb-89f3-c5e56f40d442 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:43.183376 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab436ec-0cc1-47eb-89f3-c5e56f40d442 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:43.183681 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab436ec-0cc1-47eb-89f3-c5e56f40d442 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:43.183954 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab436ec-0cc1-47eb-89f3-c5e56f40d442 None admin] PATH_INFO: `/v3/domains` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:43.184654 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab436ec-0cc1-47eb-89f3-c5e56f40d442 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:50:43.184961 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab436ec-0cc1-47eb-89f3-c5e56f40d442 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:43.186331 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab436ec-0cc1-47eb-89f3-c5e56f40d442 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:43.193003 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:43.208596 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 532/1068] 10.222.0.27 () {56 vars in 1109 bytes} [Fri Aug 5 19:50:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:50:43.213723 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab436ec-0cc1-47eb-89f3-c5e56f40d442 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:43.231411 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84623e04-e635-477a-aec5-37970678acf9 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:43.288389 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19c7420-44dd-4605-96ba-1bc066ac1766 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:43.289358 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19c7420-44dd-4605-96ba-1bc066ac1766 None admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:43.289531 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19c7420-44dd-4605-96ba-1bc066ac1766 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:43.289863 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19c7420-44dd-4605-96ba-1bc066ac1766 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:43.290581 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f19c7420-44dd-4605-96ba-1bc066ac1766 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:43.292040 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f19c7420-44dd-4605-96ba-1bc066ac1766 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:43.324535 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 537/1069] 10.222.0.27 () {60 vars in 1144 bytes} [Fri Aug 5 19:50:43 2022] POST /identity/v3/endpoints => generated 327 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 05 19:50:43.373314 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab436ec-0cc1-47eb-89f3-c5e56f40d442 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:43.422247 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c176fe8-909f-4b9c-9b80-978ed1addc25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:43.422838 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c176fe8-909f-4b9c-9b80-978ed1addc25 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:43.423099 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c176fe8-909f-4b9c-9b80-978ed1addc25 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:43.423366 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c176fe8-909f-4b9c-9b80-978ed1addc25 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:43.432428 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c176fe8-909f-4b9c-9b80-978ed1addc25 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:43.434512 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c176fe8-909f-4b9c-9b80-978ed1addc25 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:43.447652 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application [None req-2c176fe8-909f-4b9c-9b80-978ed1addc25 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 19:50:43.447652 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:50:43.447652 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:50:43.447652 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:50:43.447652 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:50:43.447652 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:50:43.447652 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:50:43.447652 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:50:43.447652 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:50:43.447652 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:50:43.447652 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:50:43.447652 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:50:43.447652 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 05 19:50:43.447652 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 05 19:50:43.447652 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 05 19:50:43.454016 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 05 19:50:43.454016 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:50:43.454016 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:50:43.454016 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:50:43.454016 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:50:43.454016 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:50:43.454016 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:50:43.454016 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:50:43.454016 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application with Lock( Aug 05 19:50:43.454016 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:50:43.454016 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:50:43.454016 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:50:43.454016 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:50:43.454016 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:50:43.454016 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:50:43.454016 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:50:43.454016 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:50:43.455075 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 05 19:50:43.455075 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 05 19:50:43.455075 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 05 19:50:43.455075 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 05 19:50:43.455075 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 05 19:50:43.455075 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 05 19:50:43.455075 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 19:50:43.455075 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Aug 05 19:50:43.455075 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 533/1070] 10.222.0.27 () {56 vars in 1111 bytes} [Fri Aug 5 19:50:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 05 19:50:43.457685 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19c7420-44dd-4605-96ba-1bc066ac1766 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:43.587730 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19c7420-44dd-4605-96ba-1bc066ac1766 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:43.617807 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:43.630457 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88d5971-7a37-41e9-85b2-536b7c16600e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:43.631089 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88d5971-7a37-41e9-85b2-536b7c16600e None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:43.631383 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88d5971-7a37-41e9-85b2-536b7c16600e None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:43.631796 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88d5971-7a37-41e9-85b2-536b7c16600e None admin] PATH_INFO: `/v3/projects` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:43.632647 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d88d5971-7a37-41e9-85b2-536b7c16600e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:50:43.633445 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d88d5971-7a37-41e9-85b2-536b7c16600e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:43.635462 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d88d5971-7a37-41e9-85b2-536b7c16600e None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:43.652594 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 538/1071] 10.222.0.27 () {56 vars in 1148 bytes} [Fri Aug 5 19:50:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 19:50:43.655993 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c176fe8-909f-4b9c-9b80-978ed1addc25 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:43.704335 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:43.777058 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c176fe8-909f-4b9c-9b80-978ed1addc25 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:43.822335 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf69e59-a684-4aad-946f-949d18432804 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:43.823533 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf69e59-a684-4aad-946f-949d18432804 None admin] REQUEST_METHOD: `PUT` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:43.824621 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf69e59-a684-4aad-946f-949d18432804 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:43.825130 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf69e59-a684-4aad-946f-949d18432804 None admin] PATH_INFO: `/v3/projects/a70a82dcfc574fad90b3f5f00abbba58/users/11001a2ad96f4ec8a35aab2c64376875/roles/31647643dd7c43aba270f63d5fe83acb` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:43.828985 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf69e59-a684-4aad-946f-949d18432804 None admin] RBAC: Authorizing `identity:create_grant(project_id=a70a82dcfc574fad90b3f5f00abbba58, user_id=11001a2ad96f4ec8a35aab2c64376875, role_id=31647643dd7c43aba270f63d5fe83acb)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:43.831813 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf69e59-a684-4aad-946f-949d18432804 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:43.854623 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 534/1072] 10.222.0.27 () {58 vars in 1439 bytes} [Fri Aug 5 19:50:43 2022] PUT /identity/v3/projects/a70a82dcfc574fad90b3f5f00abbba58/users/11001a2ad96f4ec8a35aab2c64376875/roles/31647643dd7c43aba270f63d5fe83acb => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 05 19:50:44.970617 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142f102c-e93e-433e-a90a-239a50fac99e None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:44.971153 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142f102c-e93e-433e-a90a-239a50fac99e None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:44.971153 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142f102c-e93e-433e-a90a-239a50fac99e None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:44.973774 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 539/1073] 10.222.0.27 () {54 vars in 927 bytes} [Fri Aug 5 19:50:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:50:44.980064 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de284fe3-b474-4aca-b4cb-7ee0accef7bf None None] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:44.981226 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de284fe3-b474-4aca-b4cb-7ee0accef7bf None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:44.981226 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de284fe3-b474-4aca-b4cb-7ee0accef7bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:45.001569 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.auth.core [None req-de284fe3-b474-4aca-b4cb-7ee0accef7bf None None] MFA Rules not processed for user `03d1834a64cd4a43829827cb99a72747`. Rule list: `[]` (Enabled: `True`). {{(pid=80364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:50:45.003491 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-de284fe3-b474-4aca-b4cb-7ee0accef7bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:50:45.100402 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 535/1074] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Aug 5 19:50:44 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 05 19:50:45.123646 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142f102c-e93e-433e-a90a-239a50fac99e None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:45.249124 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142f102c-e93e-433e-a90a-239a50fac99e None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:45.285678 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40567691-db4b-4d36-bb7e-6901934b3716 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:45.286325 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40567691-db4b-4d36-bb7e-6901934b3716 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:45.286607 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40567691-db4b-4d36-bb7e-6901934b3716 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:45.286884 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40567691-db4b-4d36-bb7e-6901934b3716 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:45.298341 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40567691-db4b-4d36-bb7e-6901934b3716 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 19:50:45.303257 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-40567691-db4b-4d36-bb7e-6901934b3716 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 19:50:45.303257 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:50:45.303257 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:50:45.303257 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:50:45.303257 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:50:45.303257 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:50:45.303257 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:50:45.303257 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:50:45.303257 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:50:45.303257 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:50:45.303257 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:50:45.303257 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:50:45.303257 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 19:50:45.303257 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 19:50:45.303257 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 19:50:45.303257 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 19:50:45.304475 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 19:50:45.304475 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 19:50:45.304475 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 19:50:45.304475 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 19:50:45.304475 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:50:45.304475 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:50:45.304475 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:50:45.304475 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:50:45.304475 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:50:45.304475 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:50:45.304475 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:50:45.304475 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Aug 05 19:50:45.304475 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:50:45.304475 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:50:45.304475 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:50:45.304475 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:50:45.305773 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:50:45.305773 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:50:45.305773 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:50:45.305773 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:50:45.305773 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 19:50:45.305773 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 19:50:45.305773 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 19:50:45.305773 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 19:50:45.305773 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 19:50:45.305773 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 19:50:45.305773 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 19:50:45.305773 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Aug 05 19:50:45.308588 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 540/1075] 10.222.0.27 () {56 vars in 1096 bytes} [Fri Aug 5 19:50:45 2022] GET /identity/v3/roles/admin => generated 83 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:50:45.310920 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de284fe3-b474-4aca-b4cb-7ee0accef7bf None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:45.343433 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:45.439757 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de284fe3-b474-4aca-b4cb-7ee0accef7bf None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:45.494971 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c200d580-c2e8-4c19-a04b-60702abcf872 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:45.496620 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c200d580-c2e8-4c19-a04b-60702abcf872 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:45.497126 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c200d580-c2e8-4c19-a04b-60702abcf872 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:45.497626 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c200d580-c2e8-4c19-a04b-60702abcf872 None admin] PATH_INFO: `/v3/roles` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:45.498964 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c200d580-c2e8-4c19-a04b-60702abcf872 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:50:45.499331 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c200d580-c2e8-4c19-a04b-60702abcf872 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:45.500908 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c200d580-c2e8-4c19-a04b-60702abcf872 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:45.513035 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 536/1076] 10.222.0.27 () {56 vars in 1099 bytes} [Fri Aug 5 19:50:45 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:50:45.520136 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40567691-db4b-4d36-bb7e-6901934b3716 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:45.586377 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:45.654089 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40567691-db4b-4d36-bb7e-6901934b3716 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:45.697539 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d844fa09-b563-4fb7-beca-94f1cc85191f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:45.698203 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d844fa09-b563-4fb7-beca-94f1cc85191f None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:45.698476 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d844fa09-b563-4fb7-beca-94f1cc85191f None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:45.699310 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d844fa09-b563-4fb7-beca-94f1cc85191f None admin] PATH_INFO: `/v3/users/fb6447a57a204dfca40b3f6ffd36b5cf` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:45.700564 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d844fa09-b563-4fb7-beca-94f1cc85191f None admin] RBAC: Authorizing `identity:get_user(user_id=fb6447a57a204dfca40b3f6ffd36b5cf)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:45.702011 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d844fa09-b563-4fb7-beca-94f1cc85191f None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:45.704001 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 541/1077] 10.222.0.27 () {56 vars in 1177 bytes} [Fri Aug 5 19:50:45 2022] GET /identity/v3/users/fb6447a57a204dfca40b3f6ffd36b5cf => generated 285 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:50:45.714437 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c200d580-c2e8-4c19-a04b-60702abcf872 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:45.810804 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cbc215-a6f8-45c0-9aeb-69804e3abbbe None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:45.811184 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cbc215-a6f8-45c0-9aeb-69804e3abbbe None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:45.811605 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cbc215-a6f8-45c0-9aeb-69804e3abbbe None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:45.814117 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 542/1078] 10.222.0.27 () {54 vars in 927 bytes} [Fri Aug 5 19:50:45 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 05 19:50:45.822261 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb32b825-6c88-4cdd-b3c0-4d988979dadc None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:45.822699 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb32b825-6c88-4cdd-b3c0-4d988979dadc None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:45.823165 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb32b825-6c88-4cdd-b3c0-4d988979dadc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:45.852165 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-fb32b825-6c88-4cdd-b3c0-4d988979dadc None None] MFA Rules not processed for user `03d1834a64cd4a43829827cb99a72747`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:50:45.853417 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-fb32b825-6c88-4cdd-b3c0-4d988979dadc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:50:45.875417 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:45.889166 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c200d580-c2e8-4c19-a04b-60702abcf872 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:45.942307 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f11a85-0a55-4525-bc2e-93ef2e4d5c35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:45.943011 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f11a85-0a55-4525-bc2e-93ef2e4d5c35 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:45.943283 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f11a85-0a55-4525-bc2e-93ef2e4d5c35 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:45.943563 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f11a85-0a55-4525-bc2e-93ef2e4d5c35 None admin] PATH_INFO: `/v3/projects/bf700dedec5842849da197ced9f0b0cb` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:45.945003 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f11a85-0a55-4525-bc2e-93ef2e4d5c35 None admin] RBAC: Authorizing `identity:get_project(project_id=bf700dedec5842849da197ced9f0b0cb)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:45.946626 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f11a85-0a55-4525-bc2e-93ef2e4d5c35 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:45.948473 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 537/1079] 10.222.0.27 () {56 vars in 1186 bytes} [Fri Aug 5 19:50:45 2022] GET /identity/v3/projects/bf700dedec5842849da197ced9f0b0cb => generated 311 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:50:45.954611 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f11a85-0a55-4525-bc2e-93ef2e4d5c35 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:45.955671 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 543/1080] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Aug 5 19:50:45 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 05 19:50:45.978302 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb32b825-6c88-4cdd-b3c0-4d988979dadc None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:46.033303 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:46.080857 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f11a85-0a55-4525-bc2e-93ef2e4d5c35 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:46.095518 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb32b825-6c88-4cdd-b3c0-4d988979dadc None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:46.123629 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6505c8cc-2877-49a3-8e27-acd63796fb16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:46.124393 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6505c8cc-2877-49a3-8e27-acd63796fb16 None admin] REQUEST_METHOD: `PUT` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:46.124714 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6505c8cc-2877-49a3-8e27-acd63796fb16 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:46.125000 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6505c8cc-2877-49a3-8e27-acd63796fb16 None admin] PATH_INFO: `/v3/projects/bf700dedec5842849da197ced9f0b0cb/users/fb6447a57a204dfca40b3f6ffd36b5cf/roles/c173688a960b46e5be65f0ccd6fbd734` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:46.127225 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6505c8cc-2877-49a3-8e27-acd63796fb16 None admin] RBAC: Authorizing `identity:create_grant(project_id=bf700dedec5842849da197ced9f0b0cb, user_id=fb6447a57a204dfca40b3f6ffd36b5cf, role_id=c173688a960b46e5be65f0ccd6fbd734)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:46.128741 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6505c8cc-2877-49a3-8e27-acd63796fb16 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:46.135525 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376b00d1-9a01-4c79-a9db-8535f6fda92d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:46.136931 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376b00d1-9a01-4c79-a9db-8535f6fda92d None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:46.137289 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376b00d1-9a01-4c79-a9db-8535f6fda92d None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:46.137620 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376b00d1-9a01-4c79-a9db-8535f6fda92d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:46.143671 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 538/1081] 10.222.0.27 () {58 vars in 1439 bytes} [Fri Aug 5 19:50:45 2022] PUT /identity/v3/projects/bf700dedec5842849da197ced9f0b0cb/users/fb6447a57a204dfca40b3f6ffd36b5cf/roles/c173688a960b46e5be65f0ccd6fbd734 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 05 19:50:46.144817 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea16a31-0948-4b2f-8dee-b654349efd09 None None] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:46.145073 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea16a31-0948-4b2f-8dee-b654349efd09 None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:46.145355 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea16a31-0948-4b2f-8dee-b654349efd09 None None] PATH_INFO: `/` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:46.146705 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 539/1082] 10.222.0.27 () {54 vars in 927 bytes} [Fri Aug 5 19:50:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:50:46.153598 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea303a90-c2d7-4276-88e6-81db7e8fa40f None None] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:46.153892 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea303a90-c2d7-4276-88e6-81db7e8fa40f None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:46.154183 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea303a90-c2d7-4276-88e6-81db7e8fa40f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:46.174652 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.auth.core [None req-ea303a90-c2d7-4276-88e6-81db7e8fa40f None None] MFA Rules not processed for user `03d1834a64cd4a43829827cb99a72747`. Rule list: `[]` (Enabled: `True`). {{(pid=80364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:50:46.175969 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-ea303a90-c2d7-4276-88e6-81db7e8fa40f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:50:46.199958 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:46.255226 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-376b00d1-9a01-4c79-a9db-8535f6fda92d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:46.257308 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-376b00d1-9a01-4c79-a9db-8535f6fda92d None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:46.284546 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 540/1083] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Aug 5 19:50:46 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 05 19:50:46.301722 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea303a90-c2d7-4276-88e6-81db7e8fa40f None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:46.315667 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:46.408481 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 544/1084] 10.222.0.27 () {58 vars in 1282 bytes} [Fri Aug 5 19:50:45 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 430 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 05 19:50:46.420554 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376b00d1-9a01-4c79-a9db-8535f6fda92d None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:46.446363 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea303a90-c2d7-4276-88e6-81db7e8fa40f None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:46.490159 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a701be-bea5-4d73-b371-6b8c2953fd28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:46.490930 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a701be-bea5-4d73-b371-6b8c2953fd28 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:46.491218 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a701be-bea5-4d73-b371-6b8c2953fd28 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:46.491498 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a701be-bea5-4d73-b371-6b8c2953fd28 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:46.504386 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a701be-bea5-4d73-b371-6b8c2953fd28 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 19:50:46.509445 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application [None req-52a701be-bea5-4d73-b371-6b8c2953fd28 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 05 19:50:46.509445 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:50:46.509445 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:50:46.509445 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:50:46.509445 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:50:46.509445 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:50:46.509445 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:50:46.509445 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:50:46.509445 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:50:46.509445 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:50:46.509445 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:50:46.509445 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:50:46.509445 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 19:50:46.509445 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 19:50:46.509445 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 19:50:46.509445 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 19:50:46.510489 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 19:50:46.510489 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 19:50:46.510489 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 19:50:46.510489 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 19:50:46.510489 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:50:46.510489 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:50:46.510489 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:50:46.510489 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:50:46.510489 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:50:46.510489 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:50:46.510489 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:50:46.510489 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application with Lock( Aug 05 19:50:46.510489 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:50:46.510489 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:50:46.510489 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:50:46.510489 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:50:46.511528 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:50:46.511528 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:50:46.511528 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:50:46.511528 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:50:46.511528 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 19:50:46.511528 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 19:50:46.511528 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 19:50:46.511528 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 19:50:46.511528 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 19:50:46.511528 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 19:50:46.511528 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 05 19:50:46.511528 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Aug 05 19:50:46.511528 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 541/1085] 10.222.0.27 () {56 vars in 1120 bytes} [Fri Aug 5 19:50:46 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 05 19:50:46.517440 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a701be-bea5-4d73-b371-6b8c2953fd28 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:46.541099 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:46.557270 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376b00d1-9a01-4c79-a9db-8535f6fda92d None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:46.607259 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694acbe1-4622-4f40-901e-db2be3435d17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:46.607934 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694acbe1-4622-4f40-901e-db2be3435d17 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:46.608215 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694acbe1-4622-4f40-901e-db2be3435d17 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:46.608542 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694acbe1-4622-4f40-901e-db2be3435d17 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:46.620490 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-694acbe1-4622-4f40-901e-db2be3435d17 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:46.621978 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-694acbe1-4622-4f40-901e-db2be3435d17 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:46.625818 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:46.643823 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-694acbe1-4622-4f40-901e-db2be3435d17 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:50:46.643823 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:50:46.643823 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:50:46.643823 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:50:46.643823 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:50:46.643823 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:50:46.643823 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:50:46.643823 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:50:46.643823 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:50:46.643823 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:50:46.643823 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:50:46.643823 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:50:46.643823 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 19:50:46.643823 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 19:50:46.643823 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 19:50:46.644846 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 19:50:46.644846 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:50:46.644846 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:50:46.644846 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:50:46.644846 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:50:46.644846 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:50:46.644846 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:50:46.644846 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:50:46.644846 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Aug 05 19:50:46.644846 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:50:46.644846 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:50:46.644846 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:50:46.644846 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:50:46.644846 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:50:46.644846 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:50:46.644846 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:50:46.644846 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:50:46.646718 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 19:50:46.646718 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 19:50:46.646718 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:50:46.646718 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Aug 05 19:50:46.646718 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 545/1086] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Aug 5 19:50:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 05 19:50:46.651548 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694acbe1-4622-4f40-901e-db2be3435d17 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:46.691604 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a701be-bea5-4d73-b371-6b8c2953fd28 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:46.735197 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f17e27-b52f-4470-b028-c3363be23f87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:46.735832 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f17e27-b52f-4470-b028-c3363be23f87 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:46.736117 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f17e27-b52f-4470-b028-c3363be23f87 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:46.736396 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f17e27-b52f-4470-b028-c3363be23f87 None admin] PATH_INFO: `/v3/roles` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:46.737170 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f17e27-b52f-4470-b028-c3363be23f87 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:50:46.737478 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f17e27-b52f-4470-b028-c3363be23f87 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:46.739162 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f17e27-b52f-4470-b028-c3363be23f87 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:46.749151 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 542/1087] 10.222.0.27 () {56 vars in 1115 bytes} [Fri Aug 5 19:50:46 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:50:46.755053 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f17e27-b52f-4470-b028-c3363be23f87 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:46.776838 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694acbe1-4622-4f40-901e-db2be3435d17 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:46.820363 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c96bc82-1196-45d2-aa3b-417794d7c235 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:46.821318 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c96bc82-1196-45d2-aa3b-417794d7c235 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:46.821608 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c96bc82-1196-45d2-aa3b-417794d7c235 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:46.821888 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c96bc82-1196-45d2-aa3b-417794d7c235 None admin] PATH_INFO: `/v3/domains` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:46.822615 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c96bc82-1196-45d2-aa3b-417794d7c235 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:50:46.822923 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c96bc82-1196-45d2-aa3b-417794d7c235 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:46.824586 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c96bc82-1196-45d2-aa3b-417794d7c235 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:46.838243 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 546/1088] 10.222.0.27 () {56 vars in 1109 bytes} [Fri Aug 5 19:50:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 19:50:46.849074 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:46.895462 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f17e27-b52f-4470-b028-c3363be23f87 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:46.943185 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19e1d2b-19fa-4fb5-9b50-d57db82999f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:46.943875 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19e1d2b-19fa-4fb5-9b50-d57db82999f6 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:46.944152 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19e1d2b-19fa-4fb5-9b50-d57db82999f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:46.944428 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19e1d2b-19fa-4fb5-9b50-d57db82999f6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:46.958427 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19e1d2b-19fa-4fb5-9b50-d57db82999f6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:46.959969 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19e1d2b-19fa-4fb5-9b50-d57db82999f6 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:46.977781 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application [None req-b19e1d2b-19fa-4fb5-9b50-d57db82999f6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:50:46.977781 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:50:46.977781 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:50:46.977781 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:50:46.977781 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:50:46.977781 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:50:46.977781 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:50:46.977781 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:50:46.977781 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:50:46.977781 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:50:46.977781 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:50:46.977781 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:50:46.977781 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 19:50:46.977781 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 19:50:46.977781 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 19:50:46.984849 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 19:50:46.984849 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:50:46.984849 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:50:46.984849 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:50:46.984849 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:50:46.984849 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:50:46.984849 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:50:46.984849 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:50:46.984849 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application with Lock( Aug 05 19:50:46.984849 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:50:46.984849 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:50:46.984849 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:50:46.984849 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:50:46.984849 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:50:46.984849 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:50:46.984849 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:50:46.984849 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:50:46.985900 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 19:50:46.985900 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 19:50:46.985900 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:50:46.985900 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Aug 05 19:50:46.985900 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 543/1089] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Aug 5 19:50:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 05 19:50:46.986203 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c96bc82-1196-45d2-aa3b-417794d7c235 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:47.062112 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:47.114708 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c96bc82-1196-45d2-aa3b-417794d7c235 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:47.161450 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed98d06-a711-458d-948a-c875541e1df7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:47.162219 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed98d06-a711-458d-948a-c875541e1df7 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:47.162542 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed98d06-a711-458d-948a-c875541e1df7 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:47.162933 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed98d06-a711-458d-948a-c875541e1df7 None admin] PATH_INFO: `/v3/domains` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:47.163766 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed98d06-a711-458d-948a-c875541e1df7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:50:47.164178 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed98d06-a711-458d-948a-c875541e1df7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:47.165733 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed98d06-a711-458d-948a-c875541e1df7 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:47.179113 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 547/1090] 10.222.0.27 () {56 vars in 1109 bytes} [Fri Aug 5 19:50:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:50:47.185304 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19e1d2b-19fa-4fb5-9b50-d57db82999f6 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:47.293549 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19e1d2b-19fa-4fb5-9b50-d57db82999f6 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:47.319141 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:47.331740 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d926850-f65b-4ee1-a3d7-94e9348244d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:47.332398 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d926850-f65b-4ee1-a3d7-94e9348244d4 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:47.332711 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d926850-f65b-4ee1-a3d7-94e9348244d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:47.332990 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d926850-f65b-4ee1-a3d7-94e9348244d4 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:47.341448 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d926850-f65b-4ee1-a3d7-94e9348244d4 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:47.342914 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d926850-f65b-4ee1-a3d7-94e9348244d4 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:47.354908 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application [None req-9d926850-f65b-4ee1-a3d7-94e9348244d4 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 05 19:50:47.354908 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:50:47.354908 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:50:47.354908 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:50:47.354908 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:50:47.354908 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:50:47.354908 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:50:47.354908 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:50:47.354908 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:50:47.354908 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:50:47.354908 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:50:47.354908 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:50:47.354908 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 05 19:50:47.354908 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 05 19:50:47.354908 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 05 19:50:47.354908 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 05 19:50:47.356103 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:50:47.356103 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:50:47.356103 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 05 19:50:47.356103 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 19:50:47.356103 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 05 19:50:47.356103 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 05 19:50:47.356103 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:50:47.356103 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:50:47.356103 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:50:47.356103 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:50:47.356103 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:50:47.356103 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application with Lock( Aug 05 19:50:47.356103 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:50:47.356103 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:50:47.356103 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:50:47.356103 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:50:47.356103 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:50:47.356103 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:50:47.357731 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:50:47.357731 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:50:47.357731 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 05 19:50:47.357731 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 05 19:50:47.357731 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 05 19:50:47.357731 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 05 19:50:47.357731 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 05 19:50:47.357731 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 05 19:50:47.357731 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 05 19:50:47.357731 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Aug 05 19:50:47.357731 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 544/1091] 10.222.0.27 () {56 vars in 1093 bytes} [Fri Aug 5 19:50:47 2022] GET /identity/v3/users/nova => generated 82 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:50:47.364301 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed98d06-a711-458d-948a-c875541e1df7 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:47.473806 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed98d06-a711-458d-948a-c875541e1df7 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:47.490950 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:47.510255 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1cbab0-f549-4145-8828-4355c8429b10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:47.510892 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1cbab0-f549-4145-8828-4355c8429b10 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:47.511235 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1cbab0-f549-4145-8828-4355c8429b10 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:47.511511 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1cbab0-f549-4145-8828-4355c8429b10 None admin] PATH_INFO: `/v3/users` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:47.512323 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c1cbab0-f549-4145-8828-4355c8429b10 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:50:47.512667 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c1cbab0-f549-4145-8828-4355c8429b10 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:47.514119 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c1cbab0-f549-4145-8828-4355c8429b10 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:47.533003 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 548/1092] 10.222.0.27 () {56 vars in 1133 bytes} [Fri Aug 5 19:50:47 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:50:47.538767 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d926850-f65b-4ee1-a3d7-94e9348244d4 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:47.640641 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d926850-f65b-4ee1-a3d7-94e9348244d4 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:47.675606 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c882fdb9-74c8-48df-ac7f-9266912cd6f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:47.676223 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c882fdb9-74c8-48df-ac7f-9266912cd6f4 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:47.676525 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c882fdb9-74c8-48df-ac7f-9266912cd6f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:47.676803 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c882fdb9-74c8-48df-ac7f-9266912cd6f4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:47.730659 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:47.747997 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c882fdb9-74c8-48df-ac7f-9266912cd6f4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:47.749532 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c882fdb9-74c8-48df-ac7f-9266912cd6f4 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:47.859101 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 545/1093] 10.222.0.27 () {58 vars in 1282 bytes} [Fri Aug 5 19:50:47 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 05 19:50:47.868344 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1cbab0-f549-4145-8828-4355c8429b10 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:48.000823 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1cbab0-f549-4145-8828-4355c8429b10 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:48.042473 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c721476e-a5ac-43b2-a1e0-7d4461944770 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:48.043131 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c721476e-a5ac-43b2-a1e0-7d4461944770 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:48.043416 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c721476e-a5ac-43b2-a1e0-7d4461944770 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:48.043691 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c721476e-a5ac-43b2-a1e0-7d4461944770 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:48.060241 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c721476e-a5ac-43b2-a1e0-7d4461944770 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:48.061860 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c721476e-a5ac-43b2-a1e0-7d4461944770 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:48.081528 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-c721476e-a5ac-43b2-a1e0-7d4461944770 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:50:48.081528 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:50:48.081528 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:50:48.081528 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:50:48.081528 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:50:48.081528 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:50:48.081528 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:50:48.081528 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:50:48.081528 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:50:48.081528 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:50:48.081528 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:50:48.081528 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:50:48.081528 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 19:50:48.081528 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 19:50:48.081528 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 19:50:48.083005 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 19:50:48.083005 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:50:48.083005 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:50:48.083005 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:50:48.083005 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:50:48.083005 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:50:48.083005 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:50:48.083005 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:50:48.083005 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Aug 05 19:50:48.083005 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:50:48.083005 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:50:48.083005 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:50:48.083005 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:50:48.083005 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:50:48.083005 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:50:48.083005 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:50:48.083005 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:50:48.083909 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 05 19:50:48.083909 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 19:50:48.083909 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 05 19:50:48.083909 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Aug 05 19:50:48.083909 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 549/1094] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Aug 5 19:50:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:50:48.089569 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c882fdb9-74c8-48df-ac7f-9266912cd6f4 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:48.184867 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:48.210433 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c882fdb9-74c8-48df-ac7f-9266912cd6f4 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:48.246090 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1104ac-f9d6-4d57-b707-4eff447abfc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:48.246724 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1104ac-f9d6-4d57-b707-4eff447abfc6 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:48.247003 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1104ac-f9d6-4d57-b707-4eff447abfc6 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:48.247280 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1104ac-f9d6-4d57-b707-4eff447abfc6 None admin] PATH_INFO: `/v3/domains` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:48.247985 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b1104ac-f9d6-4d57-b707-4eff447abfc6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:50:48.248292 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b1104ac-f9d6-4d57-b707-4eff447abfc6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:48.249716 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b1104ac-f9d6-4d57-b707-4eff447abfc6 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:48.262612 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 546/1095] 10.222.0.27 () {56 vars in 1109 bytes} [Fri Aug 5 19:50:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:50:48.268208 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c721476e-a5ac-43b2-a1e0-7d4461944770 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:48.288092 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ced1376-cb8a-436a-a2b3-a6f07037cb3f None None] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:48.288373 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ced1376-cb8a-436a-a2b3-a6f07037cb3f None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:48.288691 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ced1376-cb8a-436a-a2b3-a6f07037cb3f None None] PATH_INFO: `/` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:48.290055 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 547/1096] 10.222.0.27 () {54 vars in 927 bytes} [Fri Aug 5 19:50:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 05 19:50:48.295649 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6ac713-9705-415d-b972-1b1a8d598fdd None None] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:48.295939 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6ac713-9705-415d-b972-1b1a8d598fdd None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:48.296222 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6ac713-9705-415d-b972-1b1a8d598fdd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:48.296816 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:48.315829 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.auth.core [None req-fc6ac713-9705-415d-b972-1b1a8d598fdd None None] MFA Rules not processed for user `03d1834a64cd4a43829827cb99a72747`. Rule list: `[]` (Enabled: `True`). {{(pid=80364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:50:48.317084 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-fc6ac713-9705-415d-b972-1b1a8d598fdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:50:48.375440 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c721476e-a5ac-43b2-a1e0-7d4461944770 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:48.393796 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 548/1097] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Aug 5 19:50:48 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 05 19:50:48.409106 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5ad60c-f225-4794-89d3-98747d10f150 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:48.409750 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5ad60c-f225-4794-89d3-98747d10f150 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:48.410030 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5ad60c-f225-4794-89d3-98747d10f150 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:48.410306 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5ad60c-f225-4794-89d3-98747d10f150 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:48.416641 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6ac713-9705-415d-b972-1b1a8d598fdd None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:48.417012 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be5ad60c-f225-4794-89d3-98747d10f150 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:48.418981 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be5ad60c-f225-4794-89d3-98747d10f150 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:48.430408 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-be5ad60c-f225-4794-89d3-98747d10f150 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 19:50:48.430408 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:50:48.430408 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:50:48.430408 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:50:48.430408 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:50:48.430408 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:50:48.430408 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:50:48.430408 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:50:48.430408 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:50:48.430408 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:50:48.430408 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:50:48.430408 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:50:48.430408 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 05 19:50:48.430408 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 05 19:50:48.430408 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 05 19:50:48.431575 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 05 19:50:48.431575 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:50:48.431575 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:50:48.431575 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:50:48.431575 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:50:48.431575 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:50:48.431575 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:50:48.431575 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:50:48.431575 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Aug 05 19:50:48.431575 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:50:48.431575 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:50:48.431575 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:50:48.431575 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:50:48.431575 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:50:48.431575 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:50:48.431575 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:50:48.431575 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:50:48.433705 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 05 19:50:48.433705 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 05 19:50:48.433705 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 05 19:50:48.433705 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 05 19:50:48.433705 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 05 19:50:48.433705 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 05 19:50:48.433705 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 05 19:50:48.433705 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Aug 05 19:50:48.433705 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 550/1098] 10.222.0.27 () {56 vars in 1111 bytes} [Fri Aug 5 19:50:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 05 19:50:48.438076 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5ad60c-f225-4794-89d3-98747d10f150 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:48.499670 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:48.525754 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6ac713-9705-415d-b972-1b1a8d598fdd None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:48.529835 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:48.545779 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5ad60c-f225-4794-89d3-98747d10f150 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:48.561047 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a79cc0-8629-4572-a86a-e7e932245be5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:48.561707 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a79cc0-8629-4572-a86a-e7e932245be5 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:48.561990 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a79cc0-8629-4572-a86a-e7e932245be5 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:48.562271 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a79cc0-8629-4572-a86a-e7e932245be5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:48.574829 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a79cc0-8629-4572-a86a-e7e932245be5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 19:50:48.579747 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application [None req-39a79cc0-8629-4572-a86a-e7e932245be5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 19:50:48.579747 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:50:48.579747 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:50:48.579747 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:50:48.579747 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:50:48.579747 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:50:48.579747 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:50:48.579747 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:50:48.579747 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:50:48.579747 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:50:48.579747 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:50:48.579747 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:50:48.579747 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 19:50:48.579747 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 19:50:48.579747 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 19:50:48.579747 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 19:50:48.580871 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 19:50:48.580871 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 19:50:48.580871 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 19:50:48.580871 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 19:50:48.580871 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:50:48.580871 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:50:48.580871 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:50:48.580871 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:50:48.580871 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:50:48.580871 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:50:48.580871 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:50:48.580871 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application with Lock( Aug 05 19:50:48.580871 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:50:48.580871 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:50:48.580871 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:50:48.580871 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:50:48.581741 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:50:48.581741 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:50:48.581741 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:50:48.581741 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:50:48.581741 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 19:50:48.581741 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 19:50:48.581741 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 19:50:48.581741 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 19:50:48.581741 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 19:50:48.581741 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 19:50:48.581741 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 19:50:48.581741 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Aug 05 19:50:48.582410 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a658fb2f-2b16-48a1-92b0-8a6762bd7217 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:48.582410 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a658fb2f-2b16-48a1-92b0-8a6762bd7217 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:48.582573 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 549/1099] 10.222.0.27 () {56 vars in 1096 bytes} [Fri Aug 5 19:50:48 2022] GET /identity/v3/roles/admin => generated 83 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 05 19:50:48.582648 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a658fb2f-2b16-48a1-92b0-8a6762bd7217 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:48.582648 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a658fb2f-2b16-48a1-92b0-8a6762bd7217 None admin] PATH_INFO: `/v3/projects` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:48.582648 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a658fb2f-2b16-48a1-92b0-8a6762bd7217 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:50:48.582648 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a658fb2f-2b16-48a1-92b0-8a6762bd7217 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:48.583845 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a658fb2f-2b16-48a1-92b0-8a6762bd7217 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:48.587375 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a79cc0-8629-4572-a86a-e7e932245be5 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:48.605293 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 551/1100] 10.222.0.27 () {56 vars in 1148 bytes} [Fri Aug 5 19:50:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 19:50:48.614328 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a658fb2f-2b16-48a1-92b0-8a6762bd7217 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:48.711306 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a79cc0-8629-4572-a86a-e7e932245be5 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:48.745327 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a658fb2f-2b16-48a1-92b0-8a6762bd7217 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:48.754939 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60411cf9-87f6-41ef-8046-8f8820d8b974 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:48.755801 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60411cf9-87f6-41ef-8046-8f8820d8b974 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:48.756005 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60411cf9-87f6-41ef-8046-8f8820d8b974 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:48.756175 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60411cf9-87f6-41ef-8046-8f8820d8b974 None admin] PATH_INFO: `/v3/roles` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:48.756894 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60411cf9-87f6-41ef-8046-8f8820d8b974 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:50:48.757203 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60411cf9-87f6-41ef-8046-8f8820d8b974 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:48.758601 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60411cf9-87f6-41ef-8046-8f8820d8b974 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:48.760274 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:48.773905 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 550/1101] 10.222.0.27 () {56 vars in 1099 bytes} [Fri Aug 5 19:50:48 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 19:50:48.780891 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60411cf9-87f6-41ef-8046-8f8820d8b974 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:48.787930 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec7e257-2a35-4fea-811f-1fac46f2dc31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:48.788574 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec7e257-2a35-4fea-811f-1fac46f2dc31 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:48.788855 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec7e257-2a35-4fea-811f-1fac46f2dc31 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:48.789123 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec7e257-2a35-4fea-811f-1fac46f2dc31 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:48.789923 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec7e257-2a35-4fea-811f-1fac46f2dc31 None admin] RBAC: Adding query filter params (role.id=31647643dd7c43aba270f63d5fe83acb, scope.project.id=a70a82dcfc574fad90b3f5f00abbba58, user.id=11001a2ad96f4ec8a35aab2c64376875) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:50:48.790230 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec7e257-2a35-4fea-811f-1fac46f2dc31 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:48.791706 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec7e257-2a35-4fea-811f-1fac46f2dc31 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:48.798095 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 552/1102] 10.222.0.27 () {56 vars in 1374 bytes} [Fri Aug 5 19:50:48 2022] GET /identity/v3/role_assignments?user.id=11001a2ad96f4ec8a35aab2c64376875&scope.project.id=a70a82dcfc574fad90b3f5f00abbba58&role.id=31647643dd7c43aba270f63d5fe83acb => generated 608 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:50:48.906262 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60411cf9-87f6-41ef-8046-8f8820d8b974 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:48.953705 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563c636e-9913-4033-a63e-03c28e1aa722 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:48.954378 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563c636e-9913-4033-a63e-03c28e1aa722 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:48.954664 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563c636e-9913-4033-a63e-03c28e1aa722 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:48.954967 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563c636e-9913-4033-a63e-03c28e1aa722 None admin] PATH_INFO: `/v3/users/fb6447a57a204dfca40b3f6ffd36b5cf` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:48.956248 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-563c636e-9913-4033-a63e-03c28e1aa722 None admin] RBAC: Authorizing `identity:get_user(user_id=fb6447a57a204dfca40b3f6ffd36b5cf)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:48.957749 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-563c636e-9913-4033-a63e-03c28e1aa722 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:48.959404 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 551/1103] 10.222.0.27 () {56 vars in 1177 bytes} [Fri Aug 5 19:50:48 2022] GET /identity/v3/users/fb6447a57a204dfca40b3f6ffd36b5cf => generated 285 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 19:50:48.966182 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec7e257-2a35-4fea-811f-1fac46f2dc31 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:48.981803 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:49.094839 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec7e257-2a35-4fea-811f-1fac46f2dc31 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:49.128687 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a3a595-ced2-4cb5-9977-9f7b9d534606 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:49.129237 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a3a595-ced2-4cb5-9977-9f7b9d534606 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:49.129519 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a3a595-ced2-4cb5-9977-9f7b9d534606 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:49.129800 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a3a595-ced2-4cb5-9977-9f7b9d534606 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:49.201674 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a3a595-ced2-4cb5-9977-9f7b9d534606 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:49.203161 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a3a595-ced2-4cb5-9977-9f7b9d534606 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:49.215495 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:49.326011 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 553/1104] 10.222.0.27 () {58 vars in 1282 bytes} [Fri Aug 5 19:50:48 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 05 19:50:49.331973 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563c636e-9913-4033-a63e-03c28e1aa722 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:49.405387 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:49.455562 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563c636e-9913-4033-a63e-03c28e1aa722 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:49.496591 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eecd16c-c940-4c9e-bc55-c8d0e08ab4f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:49.497343 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eecd16c-c940-4c9e-bc55-c8d0e08ab4f1 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:49.497729 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eecd16c-c940-4c9e-bc55-c8d0e08ab4f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:49.498130 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eecd16c-c940-4c9e-bc55-c8d0e08ab4f1 None admin] PATH_INFO: `/v3/projects/bf700dedec5842849da197ced9f0b0cb` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:49.499807 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eecd16c-c940-4c9e-bc55-c8d0e08ab4f1 None admin] RBAC: Authorizing `identity:get_project(project_id=bf700dedec5842849da197ced9f0b0cb)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:49.501416 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eecd16c-c940-4c9e-bc55-c8d0e08ab4f1 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:49.503764 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 552/1105] 10.222.0.27 () {56 vars in 1186 bytes} [Fri Aug 5 19:50:49 2022] GET /identity/v3/projects/bf700dedec5842849da197ced9f0b0cb => generated 311 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:50:49.510551 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a3a595-ced2-4cb5-9977-9f7b9d534606 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:49.635347 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a3a595-ced2-4cb5-9977-9f7b9d534606 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:49.647492 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:49.677125 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b915e6e-5a52-4be6-b557-9e7583defdf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:49.677758 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b915e6e-5a52-4be6-b557-9e7583defdf6 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:49.678046 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b915e6e-5a52-4be6-b557-9e7583defdf6 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:49.678324 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b915e6e-5a52-4be6-b557-9e7583defdf6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:49.679135 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b915e6e-5a52-4be6-b557-9e7583defdf6 None admin] RBAC: Adding query filter params (role.id=c173688a960b46e5be65f0ccd6fbd734, scope.project.id=bf700dedec5842849da197ced9f0b0cb, user.id=fb6447a57a204dfca40b3f6ffd36b5cf) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:50:49.679449 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b915e6e-5a52-4be6-b557-9e7583defdf6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:49.680936 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b915e6e-5a52-4be6-b557-9e7583defdf6 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:49.688107 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 554/1106] 10.222.0.27 () {56 vars in 1374 bytes} [Fri Aug 5 19:50:49 2022] GET /identity/v3/role_assignments?user.id=fb6447a57a204dfca40b3f6ffd36b5cf&scope.project.id=bf700dedec5842849da197ced9f0b0cb&role.id=c173688a960b46e5be65f0ccd6fbd734 => generated 608 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 19:50:51.791046 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caccb6f7-f7cd-4a83-b9c3-f42a789db4a2 None None] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:51.791595 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caccb6f7-f7cd-4a83-b9c3-f42a789db4a2 None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:51.791595 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caccb6f7-f7cd-4a83-b9c3-f42a789db4a2 None None] PATH_INFO: `/` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:51.792933 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 553/1107] 10.222.0.27 () {54 vars in 927 bytes} [Fri Aug 5 19:50:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:50:51.798791 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00bfd40-3659-4fc8-aa1f-0274fb9ff7c7 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:51.799426 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00bfd40-3659-4fc8-aa1f-0274fb9ff7c7 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:51.799426 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00bfd40-3659-4fc8-aa1f-0274fb9ff7c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:51.818240 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-a00bfd40-3659-4fc8-aa1f-0274fb9ff7c7 None None] MFA Rules not processed for user `03d1834a64cd4a43829827cb99a72747`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:50:51.819465 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-a00bfd40-3659-4fc8-aa1f-0274fb9ff7c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:50:51.890604 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 555/1108] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Aug 5 19:50:51 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 05 19:50:51.915357 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caccb6f7-f7cd-4a83-b9c3-f42a789db4a2 None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:52.025767 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caccb6f7-f7cd-4a83-b9c3-f42a789db4a2 None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:52.055061 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:52.061416 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0bfe3e0-e9ea-47de-adc1-39a793340bba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:52.062133 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0bfe3e0-e9ea-47de-adc1-39a793340bba None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:52.062330 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0bfe3e0-e9ea-47de-adc1-39a793340bba None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:52.062600 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0bfe3e0-e9ea-47de-adc1-39a793340bba None admin] PATH_INFO: `/v3/domains/default` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:52.063771 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0bfe3e0-e9ea-47de-adc1-39a793340bba None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:52.065223 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0bfe3e0-e9ea-47de-adc1-39a793340bba None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:52.066825 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 554/1109] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Aug 5 19:50:51 2022] GET /identity/v3/domains/default => generated 199 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:50:52.072731 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00bfd40-3659-4fc8-aa1f-0274fb9ff7c7 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:52.183394 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00bfd40-3659-4fc8-aa1f-0274fb9ff7c7 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:52.185757 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:52.220715 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1040b621-da9f-47bf-9315-c40985bfbc2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:52.221425 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1040b621-da9f-47bf-9315-c40985bfbc2a None admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:52.221714 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1040b621-da9f-47bf-9315-c40985bfbc2a None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:52.221991 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1040b621-da9f-47bf-9315-c40985bfbc2a None admin] PATH_INFO: `/v3/users` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:52.222712 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1040b621-da9f-47bf-9315-c40985bfbc2a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:52.224163 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1040b621-da9f-47bf-9315-c40985bfbc2a None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:52.247442 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 556/1110] 10.222.0.27 () {60 vars in 1132 bytes} [Fri Aug 5 19:50:52 2022] POST /identity/v3/users => generated 286 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 19:50:54.415235 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e6f963-1b51-4dfb-a821-d13a079bc52a None None] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:54.415751 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e6f963-1b51-4dfb-a821-d13a079bc52a None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:54.415751 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e6f963-1b51-4dfb-a821-d13a079bc52a None None] PATH_INFO: `/` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:54.417640 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 555/1111] 10.222.0.27 () {54 vars in 927 bytes} [Fri Aug 5 19:50:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:50:54.422812 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36bedcb-4f1a-4978-b0d7-79cd89658c70 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:54.423204 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36bedcb-4f1a-4978-b0d7-79cd89658c70 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:54.423363 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36bedcb-4f1a-4978-b0d7-79cd89658c70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:54.441647 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-f36bedcb-4f1a-4978-b0d7-79cd89658c70 None None] MFA Rules not processed for user `03d1834a64cd4a43829827cb99a72747`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:50:54.442823 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-f36bedcb-4f1a-4978-b0d7-79cd89658c70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:50:54.508142 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 557/1112] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Aug 5 19:50:54 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 05 19:50:54.531390 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e6f963-1b51-4dfb-a821-d13a079bc52a None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:54.631612 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e6f963-1b51-4dfb-a821-d13a079bc52a None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:54.664068 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45525b5c-c9bf-4ea9-8a6c-7e11018764de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:54.664723 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45525b5c-c9bf-4ea9-8a6c-7e11018764de None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:54.665006 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45525b5c-c9bf-4ea9-8a6c-7e11018764de None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:54.665283 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45525b5c-c9bf-4ea9-8a6c-7e11018764de None admin] PATH_INFO: `/v3/roles/dde0aa8028794132913bcba4d91858b8` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:54.666703 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45525b5c-c9bf-4ea9-8a6c-7e11018764de None admin] RBAC: Authorizing `identity:get_role(role_id=dde0aa8028794132913bcba4d91858b8)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:54.668683 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45525b5c-c9bf-4ea9-8a6c-7e11018764de None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:54.670050 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 556/1113] 10.222.0.27 () {56 vars in 1177 bytes} [Fri Aug 5 19:50:54 2022] GET /identity/v3/roles/dde0aa8028794132913bcba4d91858b8 => generated 223 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:50:54.676085 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36bedcb-4f1a-4978-b0d7-79cd89658c70 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:54.761674 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:54.771860 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36bedcb-4f1a-4978-b0d7-79cd89658c70 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:54.805083 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc97025d-5534-485b-9470-716be7bed4e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:54.805712 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc97025d-5534-485b-9470-716be7bed4e7 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:54.805986 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc97025d-5534-485b-9470-716be7bed4e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:54.806256 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc97025d-5534-485b-9470-716be7bed4e7 None admin] PATH_INFO: `/v3/users/9ba8a8c3ede94ded9e540b3453dc02d1` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:54.819315 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc97025d-5534-485b-9470-716be7bed4e7 None admin] RBAC: Authorizing `identity:get_user(user_id=9ba8a8c3ede94ded9e540b3453dc02d1)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:54.820786 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc97025d-5534-485b-9470-716be7bed4e7 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:54.822396 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 558/1114] 10.222.0.27 () {56 vars in 1177 bytes} [Fri Aug 5 19:50:54 2022] GET /identity/v3/users/9ba8a8c3ede94ded9e540b3453dc02d1 => generated 286 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:50:54.828798 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45525b5c-c9bf-4ea9-8a6c-7e11018764de None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:54.896397 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:54.929633 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45525b5c-c9bf-4ea9-8a6c-7e11018764de None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:54.963229 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d18675-7fc3-4760-ba09-1d025e009853 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:54.963848 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d18675-7fc3-4760-ba09-1d025e009853 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:54.964126 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d18675-7fc3-4760-ba09-1d025e009853 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:54.964398 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d18675-7fc3-4760-ba09-1d025e009853 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:55.033911 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d18675-7fc3-4760-ba09-1d025e009853 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:55.035371 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d18675-7fc3-4760-ba09-1d025e009853 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:55.103763 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:55.143394 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 557/1115] 10.222.0.27 () {58 vars in 1282 bytes} [Fri Aug 5 19:50:54 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 05 19:50:55.153846 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc97025d-5534-485b-9470-716be7bed4e7 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:55.259857 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc97025d-5534-485b-9470-716be7bed4e7 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:55.297854 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb9e791-ffe1-4497-ab49-0b70c26bafb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:55.298497 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb9e791-ffe1-4497-ab49-0b70c26bafb1 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:55.298773 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb9e791-ffe1-4497-ab49-0b70c26bafb1 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:55.299160 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb9e791-ffe1-4497-ab49-0b70c26bafb1 None admin] PATH_INFO: `/v3/projects/bf700dedec5842849da197ced9f0b0cb` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:55.300319 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb9e791-ffe1-4497-ab49-0b70c26bafb1 None admin] RBAC: Authorizing `identity:get_project(project_id=bf700dedec5842849da197ced9f0b0cb)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:55.301784 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb9e791-ffe1-4497-ab49-0b70c26bafb1 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:55.303444 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 559/1116] 10.222.0.27 () {56 vars in 1186 bytes} [Fri Aug 5 19:50:55 2022] GET /identity/v3/projects/bf700dedec5842849da197ced9f0b0cb => generated 311 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:50:55.310323 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d18675-7fc3-4760-ba09-1d025e009853 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:55.408358 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d18675-7fc3-4760-ba09-1d025e009853 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:55.441503 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcbcf28f-f128-4869-bd96-4f6a09bdd2fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:55.442183 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbcf28f-f128-4869-bd96-4f6a09bdd2fe None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:55.442384 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbcf28f-f128-4869-bd96-4f6a09bdd2fe None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:55.442657 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbcf28f-f128-4869-bd96-4f6a09bdd2fe None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:55.443446 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcbcf28f-f128-4869-bd96-4f6a09bdd2fe None admin] RBAC: Adding query filter params (role.id=dde0aa8028794132913bcba4d91858b8, scope.project.id=bf700dedec5842849da197ced9f0b0cb, user.id=9ba8a8c3ede94ded9e540b3453dc02d1) {{(pid=80364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:50:55.443757 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcbcf28f-f128-4869-bd96-4f6a09bdd2fe None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:55.445232 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcbcf28f-f128-4869-bd96-4f6a09bdd2fe None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:55.450624 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 558/1117] 10.222.0.27 () {56 vars in 1374 bytes} [Fri Aug 5 19:50:55 2022] GET /identity/v3/role_assignments?user.id=9ba8a8c3ede94ded9e540b3453dc02d1&scope.project.id=bf700dedec5842849da197ced9f0b0cb&role.id=dde0aa8028794132913bcba4d91858b8 => generated 259 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:50:57.456048 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe6e233-0ede-45bf-ba8d-6cbe55aeb3c3 None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:57.456613 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe6e233-0ede-45bf-ba8d-6cbe55aeb3c3 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:57.456613 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe6e233-0ede-45bf-ba8d-6cbe55aeb3c3 None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:57.457965 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 560/1118] 10.222.0.27 () {54 vars in 927 bytes} [Fri Aug 5 19:50:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:50:57.464373 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3927af6a-0173-4528-addd-f5a461e77096 None None] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:57.464901 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3927af6a-0173-4528-addd-f5a461e77096 None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:57.464991 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3927af6a-0173-4528-addd-f5a461e77096 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:57.482861 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.auth.core [None req-3927af6a-0173-4528-addd-f5a461e77096 None None] MFA Rules not processed for user `03d1834a64cd4a43829827cb99a72747`. Rule list: `[]` (Enabled: `True`). {{(pid=80364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:50:57.484070 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-3927af6a-0173-4528-addd-f5a461e77096 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:50:57.518223 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:57.564233 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 559/1119] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Aug 5 19:50:57 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 05 19:50:57.589959 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe6e233-0ede-45bf-ba8d-6cbe55aeb3c3 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:57.614082 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:57.707065 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe6e233-0ede-45bf-ba8d-6cbe55aeb3c3 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:57.742663 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666e857c-7c56-45aa-86f0-459337638fdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:57.743310 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666e857c-7c56-45aa-86f0-459337638fdd None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:57.743589 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666e857c-7c56-45aa-86f0-459337638fdd None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:57.743862 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666e857c-7c56-45aa-86f0-459337638fdd None admin] PATH_INFO: `/v3/roles/dde0aa8028794132913bcba4d91858b8` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:57.745866 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-666e857c-7c56-45aa-86f0-459337638fdd None admin] RBAC: Authorizing `identity:get_role(role_id=dde0aa8028794132913bcba4d91858b8)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:57.747235 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-666e857c-7c56-45aa-86f0-459337638fdd None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:57.748634 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 561/1120] 10.222.0.27 () {56 vars in 1177 bytes} [Fri Aug 5 19:50:57 2022] GET /identity/v3/roles/dde0aa8028794132913bcba4d91858b8 => generated 223 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:50:57.754800 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3927af6a-0173-4528-addd-f5a461e77096 None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:57.863195 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3927af6a-0173-4528-addd-f5a461e77096 None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:57.900835 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5b2041-323c-46be-88a3-93e2b444f05d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:57.901527 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5b2041-323c-46be-88a3-93e2b444f05d None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:57.901806 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5b2041-323c-46be-88a3-93e2b444f05d None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:57.902080 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5b2041-323c-46be-88a3-93e2b444f05d None admin] PATH_INFO: `/v3/users/9ba8a8c3ede94ded9e540b3453dc02d1` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:57.903293 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d5b2041-323c-46be-88a3-93e2b444f05d None admin] RBAC: Authorizing `identity:get_user(user_id=9ba8a8c3ede94ded9e540b3453dc02d1)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:57.904864 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d5b2041-323c-46be-88a3-93e2b444f05d None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:57.906480 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 560/1121] 10.222.0.27 () {56 vars in 1177 bytes} [Fri Aug 5 19:50:57 2022] GET /identity/v3/users/9ba8a8c3ede94ded9e540b3453dc02d1 => generated 286 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:50:57.912397 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666e857c-7c56-45aa-86f0-459337638fdd None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:58.008579 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:58.027923 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666e857c-7c56-45aa-86f0-459337638fdd None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:58.077051 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f996123-ee3d-476e-be45-93068ac194c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:58.077706 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f996123-ee3d-476e-be45-93068ac194c9 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:58.077995 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f996123-ee3d-476e-be45-93068ac194c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:58.078271 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f996123-ee3d-476e-be45-93068ac194c9 None admin] PATH_INFO: `/v3/projects/bf700dedec5842849da197ced9f0b0cb` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:58.079476 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f996123-ee3d-476e-be45-93068ac194c9 None admin] RBAC: Authorizing `identity:get_project(project_id=bf700dedec5842849da197ced9f0b0cb)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:58.080956 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f996123-ee3d-476e-be45-93068ac194c9 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:58.082660 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 562/1122] 10.222.0.27 () {56 vars in 1186 bytes} [Fri Aug 5 19:50:57 2022] GET /identity/v3/projects/bf700dedec5842849da197ced9f0b0cb => generated 311 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:50:58.088888 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5b2041-323c-46be-88a3-93e2b444f05d None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:50:58.123593 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:50:58.210048 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5b2041-323c-46be-88a3-93e2b444f05d None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:50:58.261384 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa652e1b-f72b-4ef4-b100-3996345b031d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:50:58.261987 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa652e1b-f72b-4ef4-b100-3996345b031d None admin] REQUEST_METHOD: `PUT` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:50:58.262273 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa652e1b-f72b-4ef4-b100-3996345b031d None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:50:58.262554 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa652e1b-f72b-4ef4-b100-3996345b031d None admin] PATH_INFO: `/v3/projects/bf700dedec5842849da197ced9f0b0cb/users/9ba8a8c3ede94ded9e540b3453dc02d1/roles/dde0aa8028794132913bcba4d91858b8` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:50:58.265072 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa652e1b-f72b-4ef4-b100-3996345b031d None admin] RBAC: Authorizing `identity:create_grant(project_id=bf700dedec5842849da197ced9f0b0cb, user_id=9ba8a8c3ede94ded9e540b3453dc02d1, role_id=dde0aa8028794132913bcba4d91858b8)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:50:58.266578 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa652e1b-f72b-4ef4-b100-3996345b031d None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:50:58.290297 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 561/1123] 10.222.0.27 () {58 vars in 1439 bytes} [Fri Aug 5 19:50:58 2022] PUT /identity/v3/projects/bf700dedec5842849da197ced9f0b0cb/users/9ba8a8c3ede94ded9e540b3453dc02d1/roles/dde0aa8028794132913bcba4d91858b8 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 19:51:00.362975 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f49dc7-246e-4123-8135-12b8d0811627 None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:51:00.363521 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f49dc7-246e-4123-8135-12b8d0811627 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:51:00.363953 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f49dc7-246e-4123-8135-12b8d0811627 None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:51:00.365464 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 563/1124] 10.222.0.27 () {54 vars in 927 bytes} [Fri Aug 5 19:51:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:51:00.370718 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae84bb29-5d60-42eb-89d8-3b13b616715a None None] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:51:00.371121 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae84bb29-5d60-42eb-89d8-3b13b616715a None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:51:00.371311 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae84bb29-5d60-42eb-89d8-3b13b616715a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:51:00.389367 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.auth.core [None req-ae84bb29-5d60-42eb-89d8-3b13b616715a None None] MFA Rules not processed for user `03d1834a64cd4a43829827cb99a72747`. Rule list: `[]` (Enabled: `True`). {{(pid=80364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:51:00.390535 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-ae84bb29-5d60-42eb-89d8-3b13b616715a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:51:00.455147 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 562/1125] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Aug 5 19:51:00 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 05 19:51:00.478907 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f49dc7-246e-4123-8135-12b8d0811627 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:51:00.582712 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f49dc7-246e-4123-8135-12b8d0811627 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:51:00.618623 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb6c473-42c4-49c6-ab62-0eae7b68ffbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:51:00.619271 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb6c473-42c4-49c6-ab62-0eae7b68ffbb None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:51:00.619553 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb6c473-42c4-49c6-ab62-0eae7b68ffbb None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:51:00.619848 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb6c473-42c4-49c6-ab62-0eae7b68ffbb None admin] PATH_INFO: `/v3/roles/dde0aa8028794132913bcba4d91858b8` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:51:00.621337 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb6c473-42c4-49c6-ab62-0eae7b68ffbb None admin] RBAC: Authorizing `identity:get_role(role_id=dde0aa8028794132913bcba4d91858b8)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:51:00.622707 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb6c473-42c4-49c6-ab62-0eae7b68ffbb None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:51:00.624082 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 564/1126] 10.222.0.27 () {56 vars in 1177 bytes} [Fri Aug 5 19:51:00 2022] GET /identity/v3/roles/dde0aa8028794132913bcba4d91858b8 => generated 223 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:51:00.630687 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae84bb29-5d60-42eb-89d8-3b13b616715a None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:51:00.653631 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:51:00.743341 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae84bb29-5d60-42eb-89d8-3b13b616715a None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:51:00.781114 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43fcfcfd-a74b-4f44-bb83-a9922d30f944 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:51:00.781762 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fcfcfd-a74b-4f44-bb83-a9922d30f944 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:51:00.782042 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fcfcfd-a74b-4f44-bb83-a9922d30f944 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:51:00.782311 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fcfcfd-a74b-4f44-bb83-a9922d30f944 None admin] PATH_INFO: `/v3/users/9ba8a8c3ede94ded9e540b3453dc02d1` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:51:00.783534 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43fcfcfd-a74b-4f44-bb83-a9922d30f944 None admin] RBAC: Authorizing `identity:get_user(user_id=9ba8a8c3ede94ded9e540b3453dc02d1)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:51:00.784976 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43fcfcfd-a74b-4f44-bb83-a9922d30f944 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:51:00.786615 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 563/1127] 10.222.0.27 () {56 vars in 1177 bytes} [Fri Aug 5 19:51:00 2022] GET /identity/v3/users/9ba8a8c3ede94ded9e540b3453dc02d1 => generated 286 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:51:00.792529 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb6c473-42c4-49c6-ab62-0eae7b68ffbb None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:51:00.814350 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:51:00.907145 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb6c473-42c4-49c6-ab62-0eae7b68ffbb None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:51:00.945977 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23137d51-1c24-43fd-9dfc-2f04000cfca6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:51:00.946598 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23137d51-1c24-43fd-9dfc-2f04000cfca6 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:51:00.946882 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23137d51-1c24-43fd-9dfc-2f04000cfca6 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:51:00.947152 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23137d51-1c24-43fd-9dfc-2f04000cfca6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:51:01.027032 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23137d51-1c24-43fd-9dfc-2f04000cfca6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:51:01.028535 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23137d51-1c24-43fd-9dfc-2f04000cfca6 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:51:01.050932 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:51:01.141167 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 565/1128] 10.222.0.27 () {58 vars in 1282 bytes} [Fri Aug 5 19:51:00 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 05 19:51:01.147890 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43fcfcfd-a74b-4f44-bb83-a9922d30f944 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:51:01.245672 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:51:01.262446 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43fcfcfd-a74b-4f44-bb83-a9922d30f944 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:51:01.301950 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2590715c-54e0-4e37-ac69-5a841fb4934c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:51:01.302622 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2590715c-54e0-4e37-ac69-5a841fb4934c None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:51:01.302903 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2590715c-54e0-4e37-ac69-5a841fb4934c None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:51:01.303180 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2590715c-54e0-4e37-ac69-5a841fb4934c None admin] PATH_INFO: `/v3/projects/bf700dedec5842849da197ced9f0b0cb` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:51:01.304447 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2590715c-54e0-4e37-ac69-5a841fb4934c None admin] RBAC: Authorizing `identity:get_project(project_id=bf700dedec5842849da197ced9f0b0cb)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:51:01.305960 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2590715c-54e0-4e37-ac69-5a841fb4934c None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:51:01.308273 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 564/1129] 10.222.0.27 () {56 vars in 1186 bytes} [Fri Aug 5 19:51:01 2022] GET /identity/v3/projects/bf700dedec5842849da197ced9f0b0cb => generated 311 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:51:01.314736 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23137d51-1c24-43fd-9dfc-2f04000cfca6 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:51:01.453313 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23137d51-1c24-43fd-9dfc-2f04000cfca6 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:51:01.486370 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:51:01.501560 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b297d5bf-be0a-4089-970b-59aa12ce5fbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:51:01.502110 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b297d5bf-be0a-4089-970b-59aa12ce5fbd None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:51:01.502387 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b297d5bf-be0a-4089-970b-59aa12ce5fbd None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:51:01.502660 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b297d5bf-be0a-4089-970b-59aa12ce5fbd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:51:01.503470 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b297d5bf-be0a-4089-970b-59aa12ce5fbd None admin] RBAC: Adding query filter params (role.id=dde0aa8028794132913bcba4d91858b8, scope.project.id=bf700dedec5842849da197ced9f0b0cb, user.id=9ba8a8c3ede94ded9e540b3453dc02d1) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:51:01.503774 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b297d5bf-be0a-4089-970b-59aa12ce5fbd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:51:01.505276 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b297d5bf-be0a-4089-970b-59aa12ce5fbd None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:51:01.511855 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 566/1130] 10.222.0.27 () {56 vars in 1374 bytes} [Fri Aug 5 19:51:01 2022] GET /identity/v3/role_assignments?user.id=9ba8a8c3ede94ded9e540b3453dc02d1&scope.project.id=bf700dedec5842849da197ced9f0b0cb&role.id=dde0aa8028794132913bcba4d91858b8 => generated 608 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:51:03.589557 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d6ed97-674d-42a6-bbfa-fc2292d3b56c None None] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:51:03.590354 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d6ed97-674d-42a6-bbfa-fc2292d3b56c None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:51:03.590354 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d6ed97-674d-42a6-bbfa-fc2292d3b56c None None] PATH_INFO: `/` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:51:03.591450 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 565/1131] 10.222.0.27 () {54 vars in 927 bytes} [Fri Aug 5 19:51:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:51:03.597249 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b0347b-3b9d-450d-a01e-d4c839efd02a None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:51:03.597956 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b0347b-3b9d-450d-a01e-d4c839efd02a None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:51:03.597956 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b0347b-3b9d-450d-a01e-d4c839efd02a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:51:03.616917 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-e7b0347b-3b9d-450d-a01e-d4c839efd02a None None] MFA Rules not processed for user `03d1834a64cd4a43829827cb99a72747`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:51:03.618127 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-e7b0347b-3b9d-450d-a01e-d4c839efd02a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:51:03.694918 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 567/1132] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Aug 5 19:51:03 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 05 19:51:03.718281 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d6ed97-674d-42a6-bbfa-fc2292d3b56c None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:51:03.819976 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d6ed97-674d-42a6-bbfa-fc2292d3b56c None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:51:03.854716 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57865a9f-1ab2-4a55-b8ff-ee852819fcb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:51:03.855341 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57865a9f-1ab2-4a55-b8ff-ee852819fcb1 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:51:03.855613 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57865a9f-1ab2-4a55-b8ff-ee852819fcb1 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:51:03.855877 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57865a9f-1ab2-4a55-b8ff-ee852819fcb1 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:51:03.857038 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57865a9f-1ab2-4a55-b8ff-ee852819fcb1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:51:03.858453 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57865a9f-1ab2-4a55-b8ff-ee852819fcb1 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:51:03.860017 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 566/1133] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Aug 5 19:51:03 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:51:03.865994 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b0347b-3b9d-450d-a01e-d4c839efd02a None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:51:03.966062 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b0347b-3b9d-450d-a01e-d4c839efd02a None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:51:03.982512 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:51:04.002847 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae19ae37-2030-4035-b803-672e36ed9675 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:51:04.003471 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae19ae37-2030-4035-b803-672e36ed9675 None admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:51:04.003747 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae19ae37-2030-4035-b803-672e36ed9675 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:51:04.004018 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae19ae37-2030-4035-b803-672e36ed9675 None admin] PATH_INFO: `/v3/projects` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:51:04.004737 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae19ae37-2030-4035-b803-672e36ed9675 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:51:04.006219 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae19ae37-2030-4035-b803-672e36ed9675 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:51:04.034961 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 568/1134] 10.222.0.27 () {60 vars in 1141 bytes} [Fri Aug 5 19:51:03 2022] POST /identity/v3/projects => generated 311 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 19:51:06.093754 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cd4ce8-2e26-4e5d-813f-22275382a51e None None] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:51:06.094288 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cd4ce8-2e26-4e5d-813f-22275382a51e None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:51:06.094288 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cd4ce8-2e26-4e5d-813f-22275382a51e None None] PATH_INFO: `/` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:51:06.095622 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 567/1135] 10.222.0.27 () {54 vars in 927 bytes} [Fri Aug 5 19:51:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:51:06.101391 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2560cd9-6a3b-4377-8ed1-2978a1a97c04 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:51:06.102136 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2560cd9-6a3b-4377-8ed1-2978a1a97c04 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:51:06.102136 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2560cd9-6a3b-4377-8ed1-2978a1a97c04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:51:06.124664 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-a2560cd9-6a3b-4377-8ed1-2978a1a97c04 None None] MFA Rules not processed for user `03d1834a64cd4a43829827cb99a72747`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:51:06.125883 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-a2560cd9-6a3b-4377-8ed1-2978a1a97c04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:51:06.198175 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 569/1136] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Aug 5 19:51:06 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 05 19:51:06.223611 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15cd4ce8-2e26-4e5d-813f-22275382a51e None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:51:06.258254 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:51:06.363262 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15cd4ce8-2e26-4e5d-813f-22275382a51e None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:51:06.414476 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefee10d-15b2-41b7-abc6-436843c5b018 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:51:06.415125 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefee10d-15b2-41b7-abc6-436843c5b018 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:51:06.415407 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefee10d-15b2-41b7-abc6-436843c5b018 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:51:06.415681 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefee10d-15b2-41b7-abc6-436843c5b018 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:51:06.416895 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eefee10d-15b2-41b7-abc6-436843c5b018 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:51:06.418337 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eefee10d-15b2-41b7-abc6-436843c5b018 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:51:06.420052 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 568/1137] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Aug 5 19:51:06 2022] GET /identity/v3/domains/default => generated 199 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:51:06.427350 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2560cd9-6a3b-4377-8ed1-2978a1a97c04 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:51:06.554745 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:51:06.562076 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2560cd9-6a3b-4377-8ed1-2978a1a97c04 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:51:06.601606 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b22089-037a-4f4a-8de3-a14b4ea057f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:51:06.602293 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b22089-037a-4f4a-8de3-a14b4ea057f9 None admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:51:06.602612 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b22089-037a-4f4a-8de3-a14b4ea057f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:51:06.602925 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b22089-037a-4f4a-8de3-a14b4ea057f9 None admin] PATH_INFO: `/v3/users` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:51:06.604197 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b22089-037a-4f4a-8de3-a14b4ea057f9 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:51:06.605741 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b22089-037a-4f4a-8de3-a14b4ea057f9 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:51:06.628647 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 570/1138] 10.222.0.27 () {60 vars in 1132 bytes} [Fri Aug 5 19:51:06 2022] POST /identity/v3/users => generated 286 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 05 19:51:08.799826 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0ffd3e-eef2-444a-9342-317a4c189837 None None] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:51:08.800372 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0ffd3e-eef2-444a-9342-317a4c189837 None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:51:08.800612 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0ffd3e-eef2-444a-9342-317a4c189837 None None] PATH_INFO: `/` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:51:08.802830 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 569/1139] 10.222.0.27 () {54 vars in 927 bytes} [Fri Aug 5 19:51:08 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:51:08.809341 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a50f0c6-9edc-470d-8b21-6326da8caf3d None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:51:08.809801 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a50f0c6-9edc-470d-8b21-6326da8caf3d None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:51:08.809958 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a50f0c6-9edc-470d-8b21-6326da8caf3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:51:08.828221 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-6a50f0c6-9edc-470d-8b21-6326da8caf3d None None] MFA Rules not processed for user `03d1834a64cd4a43829827cb99a72747`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:51:08.829540 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-6a50f0c6-9edc-470d-8b21-6326da8caf3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:51:08.902489 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 571/1140] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Aug 5 19:51:08 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 05 19:51:08.925401 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0ffd3e-eef2-444a-9342-317a4c189837 None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:51:09.020880 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:51:09.032754 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0ffd3e-eef2-444a-9342-317a4c189837 None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:51:09.078805 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f9c18d-1700-4f5c-b54f-7acf6818037f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:51:09.079501 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f9c18d-1700-4f5c-b54f-7acf6818037f None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:51:09.079871 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f9c18d-1700-4f5c-b54f-7acf6818037f None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:51:09.080240 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f9c18d-1700-4f5c-b54f-7acf6818037f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:51:09.092048 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f9c18d-1700-4f5c-b54f-7acf6818037f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 19:51:09.097199 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application [None req-73f9c18d-1700-4f5c-b54f-7acf6818037f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 19:51:09.097199 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:51:09.097199 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:51:09.097199 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:51:09.097199 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:51:09.097199 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:51:09.097199 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:51:09.097199 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:51:09.097199 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:51:09.097199 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:51:09.097199 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:51:09.097199 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:51:09.097199 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 19:51:09.097199 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 19:51:09.097199 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 19:51:09.097199 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 19:51:09.099204 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 19:51:09.099204 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 19:51:09.099204 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 19:51:09.099204 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 19:51:09.099204 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:51:09.099204 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:51:09.099204 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:51:09.099204 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:51:09.099204 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:51:09.099204 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:51:09.099204 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:51:09.099204 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application with Lock( Aug 05 19:51:09.099204 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:51:09.099204 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:51:09.099204 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:51:09.099204 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:51:09.101113 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:51:09.101113 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:51:09.101113 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:51:09.101113 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:51:09.101113 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 19:51:09.101113 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 19:51:09.101113 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 19:51:09.101113 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 19:51:09.101113 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 19:51:09.101113 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 19:51:09.101113 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 19:51:09.101113 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Aug 05 19:51:09.101113 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 570/1141] 10.222.0.27 () {56 vars in 1096 bytes} [Fri Aug 5 19:51:08 2022] GET /identity/v3/roles/admin => generated 83 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:51:09.107179 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a50f0c6-9edc-470d-8b21-6326da8caf3d None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:51:09.194688 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:51:09.216396 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a50f0c6-9edc-470d-8b21-6326da8caf3d None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:51:09.252610 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0568cf0c-3819-4747-837d-bad9191bcd75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:51:09.253237 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0568cf0c-3819-4747-837d-bad9191bcd75 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:51:09.253521 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0568cf0c-3819-4747-837d-bad9191bcd75 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:51:09.253799 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0568cf0c-3819-4747-837d-bad9191bcd75 None admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:51:09.254520 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0568cf0c-3819-4747-837d-bad9191bcd75 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:51:09.254829 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0568cf0c-3819-4747-837d-bad9191bcd75 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:51:09.256204 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0568cf0c-3819-4747-837d-bad9191bcd75 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:51:09.265175 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 572/1142] 10.222.0.27 () {56 vars in 1099 bytes} [Fri Aug 5 19:51:09 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:51:09.271158 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f9c18d-1700-4f5c-b54f-7acf6818037f None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:51:09.392389 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f9c18d-1700-4f5c-b54f-7acf6818037f None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:51:09.434862 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff551833-c68d-42ec-961c-d418288b4311 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:51:09.435568 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff551833-c68d-42ec-961c-d418288b4311 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:51:09.435948 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff551833-c68d-42ec-961c-d418288b4311 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:51:09.436267 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff551833-c68d-42ec-961c-d418288b4311 None admin] PATH_INFO: `/v3/users/ec44c9330d884942a7c807231281a85f` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:51:09.451014 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff551833-c68d-42ec-961c-d418288b4311 None admin] RBAC: Authorizing `identity:get_user(user_id=ec44c9330d884942a7c807231281a85f)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:51:09.452724 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff551833-c68d-42ec-961c-d418288b4311 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:51:09.454424 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 571/1143] 10.222.0.27 () {56 vars in 1177 bytes} [Fri Aug 5 19:51:09 2022] GET /identity/v3/users/ec44c9330d884942a7c807231281a85f => generated 286 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:51:09.460438 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0568cf0c-3819-4747-837d-bad9191bcd75 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:51:09.565009 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0568cf0c-3819-4747-837d-bad9191bcd75 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:51:09.599350 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61d0023-b0a8-4a6b-b3a8-059b31688aca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:51:09.599965 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61d0023-b0a8-4a6b-b3a8-059b31688aca None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:51:09.600242 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61d0023-b0a8-4a6b-b3a8-059b31688aca None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:51:09.600530 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61d0023-b0a8-4a6b-b3a8-059b31688aca None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:51:09.625654 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:51:09.702564 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61d0023-b0a8-4a6b-b3a8-059b31688aca None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:51:09.704116 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61d0023-b0a8-4a6b-b3a8-059b31688aca None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:51:09.831599 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 573/1144] 10.222.0.27 () {58 vars in 1282 bytes} [Fri Aug 5 19:51:09 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 05 19:51:09.838806 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff551833-c68d-42ec-961c-d418288b4311 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:51:09.854550 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:51:09.975406 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff551833-c68d-42ec-961c-d418288b4311 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:51:10.013231 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43446a61-2ed1-4db6-93a2-3e6635b9532c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:51:10.013884 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43446a61-2ed1-4db6-93a2-3e6635b9532c None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:51:10.014163 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43446a61-2ed1-4db6-93a2-3e6635b9532c None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:51:10.014435 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43446a61-2ed1-4db6-93a2-3e6635b9532c None admin] PATH_INFO: `/v3/projects/2e77bc54a55d414b98368a246c7fd7d3` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:51:10.015661 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43446a61-2ed1-4db6-93a2-3e6635b9532c None admin] RBAC: Authorizing `identity:get_project(project_id=2e77bc54a55d414b98368a246c7fd7d3)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:51:10.017162 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43446a61-2ed1-4db6-93a2-3e6635b9532c None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:51:10.018893 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 572/1145] 10.222.0.27 () {56 vars in 1186 bytes} [Fri Aug 5 19:51:09 2022] GET /identity/v3/projects/2e77bc54a55d414b98368a246c7fd7d3 => generated 311 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:51:10.026258 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61d0023-b0a8-4a6b-b3a8-059b31688aca None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:51:10.105797 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:51:10.191503 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61d0023-b0a8-4a6b-b3a8-059b31688aca None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:51:10.249105 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee55a99-a415-4ef9-b9f9-858837e419c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:51:10.249787 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee55a99-a415-4ef9-b9f9-858837e419c9 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:51:10.250147 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee55a99-a415-4ef9-b9f9-858837e419c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:51:10.250533 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee55a99-a415-4ef9-b9f9-858837e419c9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:51:10.254576 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee55a99-a415-4ef9-b9f9-858837e419c9 None admin] RBAC: Adding query filter params (role.id=c173688a960b46e5be65f0ccd6fbd734, scope.project.id=2e77bc54a55d414b98368a246c7fd7d3, user.id=ec44c9330d884942a7c807231281a85f) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:51:10.255061 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee55a99-a415-4ef9-b9f9-858837e419c9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:51:10.259289 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee55a99-a415-4ef9-b9f9-858837e419c9 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:51:10.271094 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 574/1146] 10.222.0.27 () {56 vars in 1374 bytes} [Fri Aug 5 19:51:10 2022] GET /identity/v3/role_assignments?user.id=ec44c9330d884942a7c807231281a85f&scope.project.id=2e77bc54a55d414b98368a246c7fd7d3&role.id=c173688a960b46e5be65f0ccd6fbd734 => generated 259 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:51:12.300641 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1fefc0-876e-42c3-bbea-b8f79c153d48 None None] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:51:12.301179 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1fefc0-876e-42c3-bbea-b8f79c153d48 None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:51:12.301272 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1fefc0-876e-42c3-bbea-b8f79c153d48 None None] PATH_INFO: `/` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:51:12.302641 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 573/1147] 10.222.0.27 () {54 vars in 927 bytes} [Fri Aug 5 19:51:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 05 19:51:12.309943 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2570477-2a50-4f50-a226-853e231c6327 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:51:12.310476 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2570477-2a50-4f50-a226-853e231c6327 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:51:12.310882 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2570477-2a50-4f50-a226-853e231c6327 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:51:12.339164 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-a2570477-2a50-4f50-a226-853e231c6327 None None] MFA Rules not processed for user `03d1834a64cd4a43829827cb99a72747`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:51:12.341094 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-a2570477-2a50-4f50-a226-853e231c6327 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:51:12.447102 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 575/1148] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Aug 5 19:51:12 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 05 19:51:12.469991 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1fefc0-876e-42c3-bbea-b8f79c153d48 None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:51:12.569864 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:51:12.594758 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1fefc0-876e-42c3-bbea-b8f79c153d48 None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:51:12.635198 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a14c2c-82fe-4747-b9c9-6b7ea91aa596 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:51:12.636266 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a14c2c-82fe-4747-b9c9-6b7ea91aa596 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:51:12.636828 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a14c2c-82fe-4747-b9c9-6b7ea91aa596 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:51:12.637310 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a14c2c-82fe-4747-b9c9-6b7ea91aa596 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:51:12.655803 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a14c2c-82fe-4747-b9c9-6b7ea91aa596 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 19:51:12.665560 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application [None req-65a14c2c-82fe-4747-b9c9-6b7ea91aa596 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 19:51:12.665560 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:51:12.665560 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:51:12.665560 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:51:12.665560 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:51:12.665560 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:51:12.665560 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:51:12.665560 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:51:12.665560 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:51:12.665560 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:51:12.665560 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:51:12.665560 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:51:12.665560 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 19:51:12.665560 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 19:51:12.665560 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 19:51:12.665560 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 19:51:12.667838 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 19:51:12.667838 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 19:51:12.667838 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 19:51:12.667838 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 19:51:12.667838 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:51:12.667838 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:51:12.667838 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:51:12.667838 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:51:12.667838 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:51:12.667838 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:51:12.667838 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:51:12.667838 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application with Lock( Aug 05 19:51:12.667838 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:51:12.667838 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:51:12.667838 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:51:12.667838 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:51:12.669502 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:51:12.669502 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:51:12.669502 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:51:12.669502 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:51:12.669502 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 19:51:12.669502 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 19:51:12.669502 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 19:51:12.669502 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 19:51:12.669502 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 19:51:12.669502 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 19:51:12.669502 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 19:51:12.669502 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Aug 05 19:51:12.669502 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 574/1149] 10.222.0.27 () {56 vars in 1096 bytes} [Fri Aug 5 19:51:12 2022] GET /identity/v3/roles/admin => generated 83 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 05 19:51:12.678638 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2570477-2a50-4f50-a226-853e231c6327 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:51:12.747405 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:51:12.842012 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2570477-2a50-4f50-a226-853e231c6327 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:51:12.877053 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7cbbeda-daa1-4a78-9fbf-78090add7fb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:51:12.877671 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7cbbeda-daa1-4a78-9fbf-78090add7fb0 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:51:12.877955 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7cbbeda-daa1-4a78-9fbf-78090add7fb0 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:51:12.878228 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7cbbeda-daa1-4a78-9fbf-78090add7fb0 None admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:51:12.878945 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7cbbeda-daa1-4a78-9fbf-78090add7fb0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:51:12.879249 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7cbbeda-daa1-4a78-9fbf-78090add7fb0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:51:12.880652 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7cbbeda-daa1-4a78-9fbf-78090add7fb0 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:51:12.888943 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 576/1150] 10.222.0.27 () {56 vars in 1099 bytes} [Fri Aug 5 19:51:12 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:51:12.895717 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a14c2c-82fe-4747-b9c9-6b7ea91aa596 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:51:13.063532 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a14c2c-82fe-4747-b9c9-6b7ea91aa596 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:51:13.122894 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4cf90b-0794-4f93-9237-139b8d292bec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:51:13.123986 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4cf90b-0794-4f93-9237-139b8d292bec None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:51:13.124478 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4cf90b-0794-4f93-9237-139b8d292bec None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:51:13.124981 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4cf90b-0794-4f93-9237-139b8d292bec None admin] PATH_INFO: `/v3/users/ec44c9330d884942a7c807231281a85f` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:51:13.126197 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf4cf90b-0794-4f93-9237-139b8d292bec None admin] RBAC: Authorizing `identity:get_user(user_id=ec44c9330d884942a7c807231281a85f)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:51:13.127627 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf4cf90b-0794-4f93-9237-139b8d292bec None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:51:13.129249 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 575/1151] 10.222.0.27 () {56 vars in 1177 bytes} [Fri Aug 5 19:51:12 2022] GET /identity/v3/users/ec44c9330d884942a7c807231281a85f => generated 286 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:51:13.137443 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7cbbeda-daa1-4a78-9fbf-78090add7fb0 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:51:13.278732 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:51:13.283895 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7cbbeda-daa1-4a78-9fbf-78090add7fb0 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:51:13.338780 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ba6888-dd39-458f-8d06-24852ed2007e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:51:13.339825 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ba6888-dd39-458f-8d06-24852ed2007e None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:51:13.340199 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ba6888-dd39-458f-8d06-24852ed2007e None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:51:13.340692 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ba6888-dd39-458f-8d06-24852ed2007e None admin] PATH_INFO: `/v3/projects/2e77bc54a55d414b98368a246c7fd7d3` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:51:13.342531 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ba6888-dd39-458f-8d06-24852ed2007e None admin] RBAC: Authorizing `identity:get_project(project_id=2e77bc54a55d414b98368a246c7fd7d3)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:51:13.344951 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ba6888-dd39-458f-8d06-24852ed2007e None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:51:13.347766 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 577/1152] 10.222.0.27 () {56 vars in 1186 bytes} [Fri Aug 5 19:51:13 2022] GET /identity/v3/projects/2e77bc54a55d414b98368a246c7fd7d3 => generated 311 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:51:13.355016 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4cf90b-0794-4f93-9237-139b8d292bec None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:51:13.371074 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:51:13.489246 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4cf90b-0794-4f93-9237-139b8d292bec None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:51:13.544574 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdca919a-6a2c-46e9-941a-12e028cf09f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:51:13.545720 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdca919a-6a2c-46e9-941a-12e028cf09f7 None admin] REQUEST_METHOD: `PUT` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:51:13.546219 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdca919a-6a2c-46e9-941a-12e028cf09f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:51:13.546718 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdca919a-6a2c-46e9-941a-12e028cf09f7 None admin] PATH_INFO: `/v3/projects/2e77bc54a55d414b98368a246c7fd7d3/users/ec44c9330d884942a7c807231281a85f/roles/c173688a960b46e5be65f0ccd6fbd734` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:51:13.551091 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdca919a-6a2c-46e9-941a-12e028cf09f7 None admin] RBAC: Authorizing `identity:create_grant(project_id=2e77bc54a55d414b98368a246c7fd7d3, user_id=ec44c9330d884942a7c807231281a85f, role_id=c173688a960b46e5be65f0ccd6fbd734)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:51:13.553969 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdca919a-6a2c-46e9-941a-12e028cf09f7 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:51:13.576346 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 576/1153] 10.222.0.27 () {58 vars in 1439 bytes} [Fri Aug 5 19:51:13 2022] PUT /identity/v3/projects/2e77bc54a55d414b98368a246c7fd7d3/users/ec44c9330d884942a7c807231281a85f/roles/c173688a960b46e5be65f0ccd6fbd734 => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 19:51:16.012036 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c555b59-1768-456e-8e12-6dd8eefcd4f8 None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:51:16.012688 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c555b59-1768-456e-8e12-6dd8eefcd4f8 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:51:16.013012 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c555b59-1768-456e-8e12-6dd8eefcd4f8 None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:51:16.015559 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 578/1154] 10.222.0.27 () {54 vars in 927 bytes} [Fri Aug 5 19:51:16 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:51:16.025163 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2dc38ea-592a-4980-a8fa-f177a6c4dc5c None None] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:51:16.026477 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2dc38ea-592a-4980-a8fa-f177a6c4dc5c None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:51:16.026642 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2dc38ea-592a-4980-a8fa-f177a6c4dc5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:51:16.052096 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.auth.core [None req-b2dc38ea-592a-4980-a8fa-f177a6c4dc5c None None] MFA Rules not processed for user `03d1834a64cd4a43829827cb99a72747`. Rule list: `[]` (Enabled: `True`). {{(pid=80364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:51:16.053543 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-b2dc38ea-592a-4980-a8fa-f177a6c4dc5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:51:16.133138 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:51:16.147261 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 577/1155] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Aug 5 19:51:16 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 05 19:51:16.184085 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c555b59-1768-456e-8e12-6dd8eefcd4f8 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:51:16.360705 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c555b59-1768-456e-8e12-6dd8eefcd4f8 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:51:16.421089 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634c0aa6-99f6-4a4d-a7e5-58b595289689 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:51:16.422119 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634c0aa6-99f6-4a4d-a7e5-58b595289689 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:51:16.422575 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634c0aa6-99f6-4a4d-a7e5-58b595289689 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:51:16.423023 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634c0aa6-99f6-4a4d-a7e5-58b595289689 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:51:16.442545 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-634c0aa6-99f6-4a4d-a7e5-58b595289689 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 19:51:16.451754 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-634c0aa6-99f6-4a4d-a7e5-58b595289689 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 19:51:16.451754 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:51:16.451754 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:51:16.451754 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:51:16.451754 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:51:16.451754 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:51:16.451754 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:51:16.451754 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:51:16.451754 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:51:16.451754 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:51:16.451754 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:51:16.451754 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:51:16.451754 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 19:51:16.451754 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 19:51:16.451754 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 19:51:16.451754 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 19:51:16.453410 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 19:51:16.453410 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 19:51:16.453410 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 19:51:16.453410 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 19:51:16.453410 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:51:16.453410 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:51:16.453410 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:51:16.453410 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:51:16.453410 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:51:16.453410 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:51:16.453410 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:51:16.453410 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Aug 05 19:51:16.453410 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:51:16.453410 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:51:16.453410 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:51:16.453410 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:51:16.454838 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:51:16.454838 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:51:16.454838 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:51:16.454838 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:51:16.454838 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 19:51:16.454838 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 19:51:16.454838 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 19:51:16.454838 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 19:51:16.454838 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 19:51:16.454838 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 19:51:16.454838 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 19:51:16.454838 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Aug 05 19:51:16.454838 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 579/1156] 10.222.0.27 () {56 vars in 1096 bytes} [Fri Aug 5 19:51:16 2022] GET /identity/v3/roles/admin => generated 83 bytes in 271 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:51:16.464329 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2dc38ea-592a-4980-a8fa-f177a6c4dc5c None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:51:16.598058 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2dc38ea-592a-4980-a8fa-f177a6c4dc5c None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:51:16.634221 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d52c027-4d81-4302-8b3c-f00e9106a01f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:51:16.634920 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d52c027-4d81-4302-8b3c-f00e9106a01f None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:51:16.635244 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d52c027-4d81-4302-8b3c-f00e9106a01f None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:51:16.635560 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d52c027-4d81-4302-8b3c-f00e9106a01f None admin] PATH_INFO: `/v3/roles` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:51:16.636325 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d52c027-4d81-4302-8b3c-f00e9106a01f None admin] RBAC: Adding query filter params (name=admin) {{(pid=80364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:51:16.636796 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d52c027-4d81-4302-8b3c-f00e9106a01f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:51:16.638233 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d52c027-4d81-4302-8b3c-f00e9106a01f None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:51:16.647668 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 578/1157] 10.222.0.27 () {56 vars in 1099 bytes} [Fri Aug 5 19:51:16 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 19:51:16.657926 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634c0aa6-99f6-4a4d-a7e5-58b595289689 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:51:16.701798 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:51:16.810992 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634c0aa6-99f6-4a4d-a7e5-58b595289689 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:51:16.847499 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0a936d-94d8-41e0-897e-31a6ad522a74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:51:16.848129 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0a936d-94d8-41e0-897e-31a6ad522a74 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:51:16.848405 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0a936d-94d8-41e0-897e-31a6ad522a74 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:51:16.848700 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0a936d-94d8-41e0-897e-31a6ad522a74 None admin] PATH_INFO: `/v3/users/ec44c9330d884942a7c807231281a85f` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:51:16.849877 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0a936d-94d8-41e0-897e-31a6ad522a74 None admin] RBAC: Authorizing `identity:get_user(user_id=ec44c9330d884942a7c807231281a85f)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:51:16.851322 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0a936d-94d8-41e0-897e-31a6ad522a74 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:51:16.852991 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 580/1158] 10.222.0.27 () {56 vars in 1177 bytes} [Fri Aug 5 19:51:16 2022] GET /identity/v3/users/ec44c9330d884942a7c807231281a85f => generated 286 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:51:16.859534 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d52c027-4d81-4302-8b3c-f00e9106a01f None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:51:16.924558 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:51:16.970029 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d52c027-4d81-4302-8b3c-f00e9106a01f None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:51:17.005081 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fdbfc37-78be-4643-a85f-1033030841ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:51:17.005703 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdbfc37-78be-4643-a85f-1033030841ed None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:51:17.005985 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdbfc37-78be-4643-a85f-1033030841ed None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:51:17.006258 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdbfc37-78be-4643-a85f-1033030841ed None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:51:17.080629 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fdbfc37-78be-4643-a85f-1033030841ed None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:51:17.082103 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fdbfc37-78be-4643-a85f-1033030841ed None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:51:17.154848 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:51:17.231154 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 579/1159] 10.222.0.27 () {58 vars in 1282 bytes} [Fri Aug 5 19:51:16 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 05 19:51:17.240916 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0a936d-94d8-41e0-897e-31a6ad522a74 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:51:17.374234 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:51:17.385875 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0a936d-94d8-41e0-897e-31a6ad522a74 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:51:17.429991 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb178370-a408-40f2-bc29-2a4cbabc9e2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:51:17.430710 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb178370-a408-40f2-bc29-2a4cbabc9e2f None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:51:17.431028 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb178370-a408-40f2-bc29-2a4cbabc9e2f None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:51:17.431930 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb178370-a408-40f2-bc29-2a4cbabc9e2f None admin] PATH_INFO: `/v3/projects/2e77bc54a55d414b98368a246c7fd7d3` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:51:17.433249 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb178370-a408-40f2-bc29-2a4cbabc9e2f None admin] RBAC: Authorizing `identity:get_project(project_id=2e77bc54a55d414b98368a246c7fd7d3)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:51:17.434749 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb178370-a408-40f2-bc29-2a4cbabc9e2f None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:51:17.436521 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 581/1160] 10.222.0.27 () {56 vars in 1186 bytes} [Fri Aug 5 19:51:17 2022] GET /identity/v3/projects/2e77bc54a55d414b98368a246c7fd7d3 => generated 311 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:51:17.443028 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fdbfc37-78be-4643-a85f-1033030841ed None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:51:17.585573 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fdbfc37-78be-4643-a85f-1033030841ed None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:51:17.623509 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6372650d-a9b4-4c16-8d16-bde7a266a745 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:51:17.624241 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6372650d-a9b4-4c16-8d16-bde7a266a745 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:51:17.624591 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6372650d-a9b4-4c16-8d16-bde7a266a745 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:51:17.624905 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6372650d-a9b4-4c16-8d16-bde7a266a745 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:51:17.625741 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6372650d-a9b4-4c16-8d16-bde7a266a745 None admin] RBAC: Adding query filter params (role.id=c173688a960b46e5be65f0ccd6fbd734, scope.project.id=2e77bc54a55d414b98368a246c7fd7d3, user.id=ec44c9330d884942a7c807231281a85f) {{(pid=80364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:51:17.626190 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6372650d-a9b4-4c16-8d16-bde7a266a745 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:51:17.627683 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6372650d-a9b4-4c16-8d16-bde7a266a745 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:51:17.634479 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 580/1161] 10.222.0.27 () {56 vars in 1374 bytes} [Fri Aug 5 19:51:17 2022] GET /identity/v3/role_assignments?user.id=ec44c9330d884942a7c807231281a85f&scope.project.id=2e77bc54a55d414b98368a246c7fd7d3&role.id=c173688a960b46e5be65f0ccd6fbd734 => generated 608 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 19:51:19.944879 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e776efbc-adac-42f7-a638-6902df91cfbd None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:51:19.945797 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e776efbc-adac-42f7-a638-6902df91cfbd None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:51:19.945963 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e776efbc-adac-42f7-a638-6902df91cfbd None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:51:19.948841 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 582/1162] 10.222.0.27 () {54 vars in 927 bytes} [Fri Aug 5 19:51:19 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:51:19.958089 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a45351-725f-412b-87ce-b889bff10dad None None] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:51:19.958871 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a45351-725f-412b-87ce-b889bff10dad None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:51:19.959020 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a45351-725f-412b-87ce-b889bff10dad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:51:19.985553 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.auth.core [None req-d1a45351-725f-412b-87ce-b889bff10dad None None] MFA Rules not processed for user `03d1834a64cd4a43829827cb99a72747`. Rule list: `[]` (Enabled: `True`). {{(pid=80364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:51:19.987373 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-d1a45351-725f-412b-87ce-b889bff10dad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:51:20.012159 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:51:20.066705 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 581/1163] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Aug 5 19:51:19 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 05 19:51:20.105902 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e776efbc-adac-42f7-a638-6902df91cfbd None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:51:20.266740 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e776efbc-adac-42f7-a638-6902df91cfbd None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:51:20.318959 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78336662-a127-4a65-b419-63d5551fbd57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:51:20.319628 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78336662-a127-4a65-b419-63d5551fbd57 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:51:20.319943 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78336662-a127-4a65-b419-63d5551fbd57 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:51:20.320256 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78336662-a127-4a65-b419-63d5551fbd57 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:51:20.370096 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:51:20.419598 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78336662-a127-4a65-b419-63d5551fbd57 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:51:20.421168 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78336662-a127-4a65-b419-63d5551fbd57 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:51:20.557163 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 583/1164] 10.222.0.27 () {58 vars in 1282 bytes} [Fri Aug 5 19:51:20 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 452 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 05 19:51:20.566669 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a45351-725f-412b-87ce-b889bff10dad None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:51:20.744183 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a45351-725f-412b-87ce-b889bff10dad None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:51:20.800345 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1adc882d-c49e-4e1a-bdfa-9a2cdd54f099 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:51:20.801440 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1adc882d-c49e-4e1a-bdfa-9a2cdd54f099 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:51:20.801925 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1adc882d-c49e-4e1a-bdfa-9a2cdd54f099 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:51:20.802405 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1adc882d-c49e-4e1a-bdfa-9a2cdd54f099 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:51:20.815325 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1adc882d-c49e-4e1a-bdfa-9a2cdd54f099 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:51:20.817818 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1adc882d-c49e-4e1a-bdfa-9a2cdd54f099 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:51:20.834803 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application [None req-1adc882d-c49e-4e1a-bdfa-9a2cdd54f099 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Aug 05 19:51:20.834803 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:51:20.834803 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Aug 05 19:51:20.834803 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Aug 05 19:51:20.834803 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 05 19:51:20.834803 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 05 19:51:20.834803 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 05 19:51:20.834803 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 05 19:51:20.834803 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Aug 05 19:51:20.834803 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Aug 05 19:51:20.834803 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 05 19:51:20.834803 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Aug 05 19:51:20.834803 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:51:20.834803 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:51:20.834803 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:51:20.834803 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:51:20.834803 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:51:20.837046 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:51:20.837046 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:51:20.837046 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:51:20.837046 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:51:20.837046 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:51:20.837046 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:51:20.837046 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 05 19:51:20.837046 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 05 19:51:20.837046 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 05 19:51:20.837046 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 05 19:51:20.837046 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:51:20.837046 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:51:20.837046 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:51:20.837046 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:51:20.837046 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:51:20.837046 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:51:20.838806 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:51:20.838806 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application with Lock( Aug 05 19:51:20.838806 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:51:20.838806 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:51:20.838806 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:51:20.838806 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:51:20.838806 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:51:20.838806 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:51:20.838806 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:51:20.838806 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:51:20.838806 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Aug 05 19:51:20.838806 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 05 19:51:20.838806 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Aug 05 19:51:20.838806 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Aug 05 19:51:20.838806 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 582/1165] 10.222.0.27 () {56 vars in 1117 bytes} [Fri Aug 5 19:51:20 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 271 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:51:20.843466 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78336662-a127-4a65-b419-63d5551fbd57 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:51:20.933368 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:51:20.974104 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78336662-a127-4a65-b419-63d5551fbd57 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:51:21.020123 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f4facd-091a-42b3-b241-f1fc0538d434 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:51:21.021163 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f4facd-091a-42b3-b241-f1fc0538d434 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:51:21.021635 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f4facd-091a-42b3-b241-f1fc0538d434 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:51:21.022102 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f4facd-091a-42b3-b241-f1fc0538d434 None admin] PATH_INFO: `/v3/domains` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:51:21.023266 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f4facd-091a-42b3-b241-f1fc0538d434 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:51:21.024043 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f4facd-091a-42b3-b241-f1fc0538d434 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:51:21.027166 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f4facd-091a-42b3-b241-f1fc0538d434 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:51:21.040293 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 584/1166] 10.222.0.27 () {56 vars in 1115 bytes} [Fri Aug 5 19:51:20 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:51:23.173174 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fa9b72-385c-4cfd-9ff5-bace343c6e5d None None] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:51:23.173741 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fa9b72-385c-4cfd-9ff5-bace343c6e5d None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:51:23.173950 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fa9b72-385c-4cfd-9ff5-bace343c6e5d None None] PATH_INFO: `/` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:51:23.175338 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 583/1167] 10.222.0.27 () {54 vars in 927 bytes} [Fri Aug 5 19:51:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:51:23.185034 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645cd824-2a39-42ce-a7eb-e67a8dee9183 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:51:23.185810 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645cd824-2a39-42ce-a7eb-e67a8dee9183 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:51:23.185966 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645cd824-2a39-42ce-a7eb-e67a8dee9183 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:51:23.209449 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-645cd824-2a39-42ce-a7eb-e67a8dee9183 None None] MFA Rules not processed for user `03d1834a64cd4a43829827cb99a72747`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:51:23.210678 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-645cd824-2a39-42ce-a7eb-e67a8dee9183 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:51:23.290952 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 585/1168] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Aug 5 19:51:23 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 05 19:51:23.314707 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58fa9b72-385c-4cfd-9ff5-bace343c6e5d None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:51:23.342207 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:51:23.454455 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58fa9b72-385c-4cfd-9ff5-bace343c6e5d None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:51:23.505793 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14d2e71-c09b-426f-8b15-f761d40965c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:51:23.506843 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14d2e71-c09b-426f-8b15-f761d40965c9 None admin] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:51:23.507319 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14d2e71-c09b-426f-8b15-f761d40965c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:51:23.507790 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14d2e71-c09b-426f-8b15-f761d40965c9 None admin] PATH_INFO: `/v3/domains` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:51:23.509150 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14d2e71-c09b-426f-8b15-f761d40965c9 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:51:23.511687 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14d2e71-c09b-426f-8b15-f761d40965c9 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:51:23.535447 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 584/1169] 10.222.0.27 () {60 vars in 1138 bytes} [Fri Aug 5 19:51:23 2022] POST /identity/v3/domains => generated 267 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 05 19:51:25.694414 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f96a7a-6249-4e6a-954a-2a1234840409 None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:51:25.694968 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f96a7a-6249-4e6a-954a-2a1234840409 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:51:25.694968 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f96a7a-6249-4e6a-954a-2a1234840409 None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:51:25.696449 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 586/1170] 10.222.0.27 () {54 vars in 927 bytes} [Fri Aug 5 19:51:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:51:25.702835 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1053e7c4-c216-4c9f-a6aa-5bd318b4f746 None None] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:51:25.703359 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1053e7c4-c216-4c9f-a6aa-5bd318b4f746 None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:51:25.703962 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1053e7c4-c216-4c9f-a6aa-5bd318b4f746 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:51:25.724164 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.auth.core [None req-1053e7c4-c216-4c9f-a6aa-5bd318b4f746 None None] MFA Rules not processed for user `03d1834a64cd4a43829827cb99a72747`. Rule list: `[]` (Enabled: `True`). {{(pid=80364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:51:25.725402 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-1053e7c4-c216-4c9f-a6aa-5bd318b4f746 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:51:25.816041 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 585/1171] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Aug 5 19:51:25 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 05 19:51:25.840472 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f96a7a-6249-4e6a-954a-2a1234840409 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:51:25.915571 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:51:25.970464 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f96a7a-6249-4e6a-954a-2a1234840409 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:51:26.016321 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a56d406-381a-43eb-b44f-08f8d5cc3a7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:51:26.017326 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a56d406-381a-43eb-b44f-08f8d5cc3a7f None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:51:26.017765 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a56d406-381a-43eb-b44f-08f8d5cc3a7f None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:51:26.018131 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a56d406-381a-43eb-b44f-08f8d5cc3a7f None admin] PATH_INFO: `/v3/domains/f5fdf96b556f4e7ab3e00f4c70df6a3f` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:51:26.034430 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a56d406-381a-43eb-b44f-08f8d5cc3a7f None admin] RBAC: Authorizing `identity:get_domain(domain_id=f5fdf96b556f4e7ab3e00f4c70df6a3f)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:51:26.036672 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a56d406-381a-43eb-b44f-08f8d5cc3a7f None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:51:26.039223 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 587/1172] 10.222.0.27 () {56 vars in 1183 bytes} [Fri Aug 5 19:51:25 2022] GET /identity/v3/domains/f5fdf96b556f4e7ab3e00f4c70df6a3f => generated 267 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 19:51:26.045875 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1053e7c4-c216-4c9f-a6aa-5bd318b4f746 None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:51:26.049617 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:51:26.153439 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1053e7c4-c216-4c9f-a6aa-5bd318b4f746 None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:51:26.193153 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f696b772-9e51-4177-8e0f-89cddd06ae79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:51:26.193884 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f696b772-9e51-4177-8e0f-89cddd06ae79 None admin] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:51:26.194276 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f696b772-9e51-4177-8e0f-89cddd06ae79 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:51:26.194602 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f696b772-9e51-4177-8e0f-89cddd06ae79 None admin] PATH_INFO: `/v3/projects` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:51:26.195330 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f696b772-9e51-4177-8e0f-89cddd06ae79 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:51:26.197421 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f696b772-9e51-4177-8e0f-89cddd06ae79 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:51:26.232657 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 586/1173] 10.222.0.27 () {60 vars in 1141 bytes} [Fri Aug 5 19:51:26 2022] POST /identity/v3/projects => generated 361 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 19:51:28.382592 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896488b9-7313-4c35-b5f0-ee967dee8895 None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:51:28.383137 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896488b9-7313-4c35-b5f0-ee967dee8895 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:51:28.383137 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896488b9-7313-4c35-b5f0-ee967dee8895 None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:51:28.385139 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 588/1174] 10.222.0.27 () {54 vars in 927 bytes} [Fri Aug 5 19:51:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 05 19:51:28.391951 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f06a7e-0fe0-4c8b-918f-edc064529a76 None None] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:51:28.392531 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f06a7e-0fe0-4c8b-918f-edc064529a76 None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:51:28.392914 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f06a7e-0fe0-4c8b-918f-edc064529a76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:51:28.421982 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.auth.core [None req-20f06a7e-0fe0-4c8b-918f-edc064529a76 None None] MFA Rules not processed for user `03d1834a64cd4a43829827cb99a72747`. Rule list: `[]` (Enabled: `True`). {{(pid=80364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:51:28.423815 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-20f06a7e-0fe0-4c8b-918f-edc064529a76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:51:28.498983 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:51:28.518968 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 587/1175] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Aug 5 19:51:28 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 05 19:51:28.548767 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896488b9-7313-4c35-b5f0-ee967dee8895 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:51:28.692703 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896488b9-7313-4c35-b5f0-ee967dee8895 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:51:28.718033 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:51:28.736233 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51908470-52bf-44b8-8604-ed9418529fc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:51:28.736904 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51908470-52bf-44b8-8604-ed9418529fc7 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:51:28.737188 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51908470-52bf-44b8-8604-ed9418529fc7 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:51:28.737455 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51908470-52bf-44b8-8604-ed9418529fc7 None admin] PATH_INFO: `/v3/domains/f5fdf96b556f4e7ab3e00f4c70df6a3f` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:51:28.738661 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51908470-52bf-44b8-8604-ed9418529fc7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=f5fdf96b556f4e7ab3e00f4c70df6a3f)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:51:28.740087 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51908470-52bf-44b8-8604-ed9418529fc7 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:51:28.741733 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 589/1176] 10.222.0.27 () {56 vars in 1183 bytes} [Fri Aug 5 19:51:28 2022] GET /identity/v3/domains/f5fdf96b556f4e7ab3e00f4c70df6a3f => generated 267 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:51:28.747874 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f06a7e-0fe0-4c8b-918f-edc064529a76 None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:51:28.854888 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f06a7e-0fe0-4c8b-918f-edc064529a76 None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:51:28.897542 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e539e7-a00d-431a-8ec1-90aa40b2c5bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:51:28.898796 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e539e7-a00d-431a-8ec1-90aa40b2c5bf None admin] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:51:28.899185 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e539e7-a00d-431a-8ec1-90aa40b2c5bf None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:51:28.900199 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e539e7-a00d-431a-8ec1-90aa40b2c5bf None admin] PATH_INFO: `/v3/users` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:51:28.901290 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e539e7-a00d-431a-8ec1-90aa40b2c5bf None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:51:28.902767 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e539e7-a00d-431a-8ec1-90aa40b2c5bf None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:51:28.929191 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 588/1177] 10.222.0.27 () {60 vars in 1132 bytes} [Fri Aug 5 19:51:28 2022] POST /identity/v3/users => generated 311 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 05 19:51:31.101424 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6217f27b-0e1b-4946-8ed3-169d05f669a1 None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:51:31.102075 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6217f27b-0e1b-4946-8ed3-169d05f669a1 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:51:31.102075 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6217f27b-0e1b-4946-8ed3-169d05f669a1 None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:51:31.103854 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 590/1178] 10.222.0.27 () {54 vars in 927 bytes} [Fri Aug 5 19:51:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:51:31.110595 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688a0982-2168-48a0-bd7e-3813968328c5 None None] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:51:31.111139 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688a0982-2168-48a0-bd7e-3813968328c5 None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:51:31.111518 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688a0982-2168-48a0-bd7e-3813968328c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:51:31.132401 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.auth.core [None req-688a0982-2168-48a0-bd7e-3813968328c5 None None] MFA Rules not processed for user `03d1834a64cd4a43829827cb99a72747`. Rule list: `[]` (Enabled: `True`). {{(pid=80364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:51:31.133687 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-688a0982-2168-48a0-bd7e-3813968328c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:51:31.176631 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:51:31.223536 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 589/1179] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Aug 5 19:51:31 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 05 19:51:31.249558 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6217f27b-0e1b-4946-8ed3-169d05f669a1 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:51:31.365393 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6217f27b-0e1b-4946-8ed3-169d05f669a1 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:51:31.415786 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35589902-3f68-491e-ba88-58e4142caf37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:51:31.416444 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35589902-3f68-491e-ba88-58e4142caf37 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:51:31.416753 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35589902-3f68-491e-ba88-58e4142caf37 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:51:31.417022 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35589902-3f68-491e-ba88-58e4142caf37 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:51:31.428895 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35589902-3f68-491e-ba88-58e4142caf37 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 19:51:31.433931 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-35589902-3f68-491e-ba88-58e4142caf37 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 19:51:31.433931 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:51:31.433931 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:51:31.433931 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:51:31.433931 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:51:31.433931 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:51:31.433931 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:51:31.433931 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:51:31.433931 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:51:31.433931 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:51:31.433931 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:51:31.433931 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:51:31.433931 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 19:51:31.433931 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 19:51:31.433931 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 19:51:31.433931 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 19:51:31.435769 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 19:51:31.435769 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 19:51:31.435769 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 19:51:31.435769 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 19:51:31.435769 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:51:31.435769 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:51:31.435769 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:51:31.435769 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:51:31.435769 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:51:31.435769 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:51:31.435769 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:51:31.435769 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Aug 05 19:51:31.435769 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:51:31.435769 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:51:31.435769 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:51:31.435769 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:51:31.437792 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:51:31.437792 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:51:31.437792 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:51:31.437792 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:51:31.437792 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 19:51:31.437792 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 19:51:31.437792 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 19:51:31.437792 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 19:51:31.437792 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 19:51:31.437792 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 19:51:31.437792 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 19:51:31.437792 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Aug 05 19:51:31.437792 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 591/1180] 10.222.0.27 () {56 vars in 1096 bytes} [Fri Aug 5 19:51:31 2022] GET /identity/v3/roles/admin => generated 83 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 05 19:51:31.443261 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688a0982-2168-48a0-bd7e-3813968328c5 None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:51:31.558312 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688a0982-2168-48a0-bd7e-3813968328c5 None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:51:31.606077 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11b8843-bb60-4b23-98f5-8d7b9316a2b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:51:31.606712 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11b8843-bb60-4b23-98f5-8d7b9316a2b7 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:51:31.606990 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11b8843-bb60-4b23-98f5-8d7b9316a2b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:51:31.607259 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11b8843-bb60-4b23-98f5-8d7b9316a2b7 None admin] PATH_INFO: `/v3/roles` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:51:31.608162 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e11b8843-bb60-4b23-98f5-8d7b9316a2b7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:51:31.608465 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e11b8843-bb60-4b23-98f5-8d7b9316a2b7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:51:31.609875 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e11b8843-bb60-4b23-98f5-8d7b9316a2b7 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:51:31.620473 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 590/1181] 10.222.0.27 () {56 vars in 1099 bytes} [Fri Aug 5 19:51:31 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:51:31.626634 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35589902-3f68-491e-ba88-58e4142caf37 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:51:31.682079 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:51:31.747151 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35589902-3f68-491e-ba88-58e4142caf37 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:51:31.794135 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10cf3e66-530c-4802-bdd2-497f05a38808 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:51:31.794821 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10cf3e66-530c-4802-bdd2-497f05a38808 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:51:31.795102 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10cf3e66-530c-4802-bdd2-497f05a38808 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:51:31.795376 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10cf3e66-530c-4802-bdd2-497f05a38808 None admin] PATH_INFO: `/v3/users/f54d05d935f7416f89ea1873b0e42bb4` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:51:31.817614 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10cf3e66-530c-4802-bdd2-497f05a38808 None admin] RBAC: Authorizing `identity:get_user(user_id=f54d05d935f7416f89ea1873b0e42bb4)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:51:31.819872 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10cf3e66-530c-4802-bdd2-497f05a38808 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:51:31.823040 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 592/1182] 10.222.0.27 () {56 vars in 1177 bytes} [Fri Aug 5 19:51:31 2022] GET /identity/v3/users/f54d05d935f7416f89ea1873b0e42bb4 => generated 311 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 19:51:31.831968 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11b8843-bb60-4b23-98f5-8d7b9316a2b7 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:51:31.872292 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:51:31.958766 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11b8843-bb60-4b23-98f5-8d7b9316a2b7 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:51:32.016019 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60395e16-2c78-46cd-ac2e-ea3d9fe973c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:51:32.016880 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60395e16-2c78-46cd-ac2e-ea3d9fe973c8 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:51:32.017162 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60395e16-2c78-46cd-ac2e-ea3d9fe973c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:51:32.017435 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60395e16-2c78-46cd-ac2e-ea3d9fe973c8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:51:32.110753 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60395e16-2c78-46cd-ac2e-ea3d9fe973c8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:51:32.113548 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60395e16-2c78-46cd-ac2e-ea3d9fe973c8 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:51:32.166661 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:51:32.243609 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 591/1183] 10.222.0.27 () {58 vars in 1282 bytes} [Fri Aug 5 19:51:31 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 414 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 05 19:51:32.253122 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10cf3e66-530c-4802-bdd2-497f05a38808 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:51:32.378591 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10cf3e66-530c-4802-bdd2-497f05a38808 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:51:32.381101 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:51:32.418121 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7ec2b3-b005-4517-bc0a-9fe7a1a8b4ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:51:32.418676 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7ec2b3-b005-4517-bc0a-9fe7a1a8b4ca None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:51:32.418947 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7ec2b3-b005-4517-bc0a-9fe7a1a8b4ca None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:51:32.419215 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7ec2b3-b005-4517-bc0a-9fe7a1a8b4ca None admin] PATH_INFO: `/v3/projects/38e470bd429b46e091a36078bfbdea70` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:51:32.420382 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df7ec2b3-b005-4517-bc0a-9fe7a1a8b4ca None admin] RBAC: Authorizing `identity:get_project(project_id=38e470bd429b46e091a36078bfbdea70)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:51:32.421830 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df7ec2b3-b005-4517-bc0a-9fe7a1a8b4ca None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:51:32.423561 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 593/1184] 10.222.0.27 () {56 vars in 1186 bytes} [Fri Aug 5 19:51:32 2022] GET /identity/v3/projects/38e470bd429b46e091a36078bfbdea70 => generated 361 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:51:32.431068 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60395e16-2c78-46cd-ac2e-ea3d9fe973c8 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:51:32.557436 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60395e16-2c78-46cd-ac2e-ea3d9fe973c8 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:51:32.588335 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:51:32.596477 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7301bf0-c16e-435f-94eb-cf0d4f694725 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:51:32.597132 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7301bf0-c16e-435f-94eb-cf0d4f694725 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:51:32.597415 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7301bf0-c16e-435f-94eb-cf0d4f694725 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:51:32.597687 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7301bf0-c16e-435f-94eb-cf0d4f694725 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:51:32.598489 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7301bf0-c16e-435f-94eb-cf0d4f694725 None admin] RBAC: Adding query filter params (role.id=c173688a960b46e5be65f0ccd6fbd734, scope.project.id=38e470bd429b46e091a36078bfbdea70, user.id=f54d05d935f7416f89ea1873b0e42bb4) {{(pid=80364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:51:32.598799 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7301bf0-c16e-435f-94eb-cf0d4f694725 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:51:32.600272 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7301bf0-c16e-435f-94eb-cf0d4f694725 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:51:32.606900 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 592/1185] 10.222.0.27 () {56 vars in 1374 bytes} [Fri Aug 5 19:51:32 2022] GET /identity/v3/role_assignments?user.id=f54d05d935f7416f89ea1873b0e42bb4&scope.project.id=38e470bd429b46e091a36078bfbdea70&role.id=c173688a960b46e5be65f0ccd6fbd734 => generated 259 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:51:34.658856 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdf2cf6-7782-4445-834a-0be2d3dd14ac None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:51:34.658856 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdf2cf6-7782-4445-834a-0be2d3dd14ac None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:51:34.658856 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdf2cf6-7782-4445-834a-0be2d3dd14ac None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:51:34.661235 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 594/1186] 10.222.0.27 () {54 vars in 927 bytes} [Fri Aug 5 19:51:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:51:34.669693 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e000d71d-2212-4aab-bf7c-ed477a1f47d2 None None] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:51:34.669693 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e000d71d-2212-4aab-bf7c-ed477a1f47d2 None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:51:34.669693 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e000d71d-2212-4aab-bf7c-ed477a1f47d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:51:34.690857 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.auth.core [None req-e000d71d-2212-4aab-bf7c-ed477a1f47d2 None None] MFA Rules not processed for user `03d1834a64cd4a43829827cb99a72747`. Rule list: `[]` (Enabled: `True`). {{(pid=80364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:51:34.692555 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-e000d71d-2212-4aab-bf7c-ed477a1f47d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:51:34.789094 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 593/1187] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Aug 5 19:51:34 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 05 19:51:34.813543 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fdf2cf6-7782-4445-834a-0be2d3dd14ac None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:51:34.943103 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fdf2cf6-7782-4445-834a-0be2d3dd14ac None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:51:34.990951 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8c6c2a-d0bc-4649-8b27-e9a7a697c2f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:51:34.991596 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8c6c2a-d0bc-4649-8b27-e9a7a697c2f2 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:51:34.991877 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8c6c2a-d0bc-4649-8b27-e9a7a697c2f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:51:34.992154 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8c6c2a-d0bc-4649-8b27-e9a7a697c2f2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:51:35.004105 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d8c6c2a-d0bc-4649-8b27-e9a7a697c2f2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 19:51:35.010302 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-9d8c6c2a-d0bc-4649-8b27-e9a7a697c2f2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 19:51:35.010302 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:51:35.010302 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:51:35.010302 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:51:35.010302 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:51:35.010302 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:51:35.010302 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:51:35.010302 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:51:35.010302 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:51:35.010302 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:51:35.010302 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:51:35.010302 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:51:35.010302 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 19:51:35.010302 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 19:51:35.010302 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 19:51:35.010302 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 19:51:35.011267 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 19:51:35.011267 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 19:51:35.011267 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 19:51:35.011267 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 19:51:35.011267 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:51:35.011267 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:51:35.011267 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:51:35.011267 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:51:35.011267 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:51:35.011267 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:51:35.011267 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:51:35.011267 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Aug 05 19:51:35.011267 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:51:35.011267 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:51:35.011267 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:51:35.011267 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:51:35.012242 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:51:35.012242 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:51:35.012242 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:51:35.012242 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:51:35.012242 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 19:51:35.012242 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 19:51:35.012242 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 19:51:35.012242 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 19:51:35.012242 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 19:51:35.012242 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 19:51:35.012242 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 19:51:35.012242 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Aug 05 19:51:35.012242 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 595/1188] 10.222.0.27 () {56 vars in 1096 bytes} [Fri Aug 5 19:51:34 2022] GET /identity/v3/roles/admin => generated 83 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:51:35.018304 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e000d71d-2212-4aab-bf7c-ed477a1f47d2 None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:51:35.133904 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e000d71d-2212-4aab-bf7c-ed477a1f47d2 None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:51:35.162977 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:51:35.173692 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d729b74-a85b-48c0-a13a-90524ddfae33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:51:35.174328 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d729b74-a85b-48c0-a13a-90524ddfae33 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:51:35.174605 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d729b74-a85b-48c0-a13a-90524ddfae33 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:51:35.174885 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d729b74-a85b-48c0-a13a-90524ddfae33 None admin] PATH_INFO: `/v3/roles` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:51:35.175608 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d729b74-a85b-48c0-a13a-90524ddfae33 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:51:35.175911 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d729b74-a85b-48c0-a13a-90524ddfae33 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:51:35.177338 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d729b74-a85b-48c0-a13a-90524ddfae33 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:51:35.186717 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 594/1189] 10.222.0.27 () {56 vars in 1099 bytes} [Fri Aug 5 19:51:35 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 19:51:35.192799 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8c6c2a-d0bc-4649-8b27-e9a7a697c2f2 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:51:35.229660 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:51:35.312338 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8c6c2a-d0bc-4649-8b27-e9a7a697c2f2 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:51:35.358596 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952acb86-f237-45c5-8220-cff7041905d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:51:35.359554 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952acb86-f237-45c5-8220-cff7041905d4 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:51:35.359962 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952acb86-f237-45c5-8220-cff7041905d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:51:35.360350 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952acb86-f237-45c5-8220-cff7041905d4 None admin] PATH_INFO: `/v3/users/f54d05d935f7416f89ea1873b0e42bb4` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:51:35.362044 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-952acb86-f237-45c5-8220-cff7041905d4 None admin] RBAC: Authorizing `identity:get_user(user_id=f54d05d935f7416f89ea1873b0e42bb4)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:51:35.363635 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-952acb86-f237-45c5-8220-cff7041905d4 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:51:35.366142 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 596/1190] 10.222.0.27 () {56 vars in 1177 bytes} [Fri Aug 5 19:51:35 2022] GET /identity/v3/users/f54d05d935f7416f89ea1873b0e42bb4 => generated 311 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 19:51:35.373918 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d729b74-a85b-48c0-a13a-90524ddfae33 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:51:35.503992 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d729b74-a85b-48c0-a13a-90524ddfae33 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:51:35.555600 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f29b85-7207-459f-8790-331f169ccb27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:51:35.556242 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f29b85-7207-459f-8790-331f169ccb27 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:51:35.556555 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f29b85-7207-459f-8790-331f169ccb27 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:51:35.556831 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f29b85-7207-459f-8790-331f169ccb27 None admin] PATH_INFO: `/v3/projects/38e470bd429b46e091a36078bfbdea70` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:51:35.557992 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f29b85-7207-459f-8790-331f169ccb27 None admin] RBAC: Authorizing `identity:get_project(project_id=38e470bd429b46e091a36078bfbdea70)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:51:35.559421 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f29b85-7207-459f-8790-331f169ccb27 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:51:35.561140 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 595/1191] 10.222.0.27 () {56 vars in 1186 bytes} [Fri Aug 5 19:51:35 2022] GET /identity/v3/projects/38e470bd429b46e091a36078bfbdea70 => generated 361 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:51:35.569163 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952acb86-f237-45c5-8220-cff7041905d4 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:51:35.689951 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:51:35.693639 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952acb86-f237-45c5-8220-cff7041905d4 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:51:35.739651 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163369a8-b2ea-4532-b171-c2e48c53f306 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:51:35.740819 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163369a8-b2ea-4532-b171-c2e48c53f306 None admin] REQUEST_METHOD: `PUT` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:51:35.741299 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163369a8-b2ea-4532-b171-c2e48c53f306 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:51:35.741767 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163369a8-b2ea-4532-b171-c2e48c53f306 None admin] PATH_INFO: `/v3/projects/38e470bd429b46e091a36078bfbdea70/users/f54d05d935f7416f89ea1873b0e42bb4/roles/c173688a960b46e5be65f0ccd6fbd734` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:51:35.745157 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-163369a8-b2ea-4532-b171-c2e48c53f306 None admin] RBAC: Authorizing `identity:create_grant(project_id=38e470bd429b46e091a36078bfbdea70, user_id=f54d05d935f7416f89ea1873b0e42bb4, role_id=c173688a960b46e5be65f0ccd6fbd734)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:51:35.747707 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-163369a8-b2ea-4532-b171-c2e48c53f306 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:51:35.769565 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 597/1192] 10.222.0.27 () {58 vars in 1439 bytes} [Fri Aug 5 19:51:35 2022] PUT /identity/v3/projects/38e470bd429b46e091a36078bfbdea70/users/f54d05d935f7416f89ea1873b0e42bb4/roles/c173688a960b46e5be65f0ccd6fbd734 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 05 19:51:37.811254 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1ce538-0f48-4bc2-84f9-860db348a1fd None None] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:51:37.811816 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1ce538-0f48-4bc2-84f9-860db348a1fd None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:51:37.811816 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1ce538-0f48-4bc2-84f9-860db348a1fd None None] PATH_INFO: `/` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:51:37.813279 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 596/1193] 10.222.0.27 () {54 vars in 927 bytes} [Fri Aug 5 19:51:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 05 19:51:37.819218 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22269f30-d8b7-4190-bce2-9a0eba3798f2 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:51:37.819738 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22269f30-d8b7-4190-bce2-9a0eba3798f2 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:51:37.819830 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22269f30-d8b7-4190-bce2-9a0eba3798f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:51:37.850870 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-22269f30-d8b7-4190-bce2-9a0eba3798f2 None None] MFA Rules not processed for user `03d1834a64cd4a43829827cb99a72747`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 05 19:51:37.853502 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-22269f30-d8b7-4190-bce2-9a0eba3798f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 05 19:51:37.932195 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 598/1194] 10.222.0.27 () {58 vars in 1024 bytes} [Fri Aug 5 19:51:37 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 05 19:51:37.970416 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1ce538-0f48-4bc2-84f9-860db348a1fd None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:51:38.045270 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:51:38.095655 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1ce538-0f48-4bc2-84f9-860db348a1fd None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:51:38.132830 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61faa1e6-4c65-476f-9caa-55f2727a53c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:51:38.133505 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61faa1e6-4c65-476f-9caa-55f2727a53c5 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:51:38.133785 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61faa1e6-4c65-476f-9caa-55f2727a53c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:51:38.134054 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61faa1e6-4c65-476f-9caa-55f2727a53c5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:51:38.145459 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61faa1e6-4c65-476f-9caa-55f2727a53c5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 05 19:51:38.151192 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application [None req-61faa1e6-4c65-476f-9caa-55f2727a53c5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 19:51:38.151192 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 05 19:51:38.151192 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 05 19:51:38.151192 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 05 19:51:38.151192 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 05 19:51:38.151192 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 05 19:51:38.151192 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 05 19:51:38.151192 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 05 19:51:38.151192 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 05 19:51:38.151192 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 05 19:51:38.151192 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 05 19:51:38.151192 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 05 19:51:38.151192 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 05 19:51:38.151192 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 05 19:51:38.151192 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 05 19:51:38.151192 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 05 19:51:38.152299 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 05 19:51:38.152299 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 05 19:51:38.152299 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 05 19:51:38.152299 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 05 19:51:38.152299 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 05 19:51:38.152299 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 05 19:51:38.152299 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 05 19:51:38.152299 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 05 19:51:38.152299 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 05 19:51:38.152299 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.get_or_create( Aug 05 19:51:38.152299 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 05 19:51:38.152299 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application with Lock( Aug 05 19:51:38.152299 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 05 19:51:38.152299 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._enter() Aug 05 19:51:38.152299 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 05 19:51:38.152299 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 05 19:51:38.158204 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 05 19:51:38.158204 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.creator() Aug 05 19:51:38.158204 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 05 19:51:38.158204 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application created_value = creator( Aug 05 19:51:38.158204 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 05 19:51:38.158204 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 05 19:51:38.158204 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 05 19:51:38.158204 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 05 19:51:38.158204 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 05 19:51:38.158204 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 05 19:51:38.158204 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 05 19:51:38.158204 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Aug 05 19:51:38.158204 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 597/1195] 10.222.0.27 () {56 vars in 1096 bytes} [Fri Aug 5 19:51:37 2022] GET /identity/v3/roles/admin => generated 83 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 05 19:51:38.159218 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22269f30-d8b7-4190-bce2-9a0eba3798f2 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:51:38.280636 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:51:38.293192 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22269f30-d8b7-4190-bce2-9a0eba3798f2 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:51:38.330540 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a226a655-9b1b-4e99-895f-c893ef3f4c29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:51:38.331170 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a226a655-9b1b-4e99-895f-c893ef3f4c29 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:51:38.331440 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a226a655-9b1b-4e99-895f-c893ef3f4c29 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:51:38.331713 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a226a655-9b1b-4e99-895f-c893ef3f4c29 None admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:51:38.332443 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a226a655-9b1b-4e99-895f-c893ef3f4c29 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:51:38.332772 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a226a655-9b1b-4e99-895f-c893ef3f4c29 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:51:38.334150 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a226a655-9b1b-4e99-895f-c893ef3f4c29 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:51:38.343914 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 599/1196] 10.222.0.27 () {56 vars in 1099 bytes} [Fri Aug 5 19:51:38 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:51:38.350005 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61faa1e6-4c65-476f-9caa-55f2727a53c5 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:51:38.479894 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61faa1e6-4c65-476f-9caa-55f2727a53c5 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:51:38.498718 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:51:38.519256 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efba97fa-ee37-4fed-b951-56ac12ad876a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:51:38.519971 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efba97fa-ee37-4fed-b951-56ac12ad876a None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:51:38.520298 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efba97fa-ee37-4fed-b951-56ac12ad876a None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:51:38.520645 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efba97fa-ee37-4fed-b951-56ac12ad876a None admin] PATH_INFO: `/v3/users/f54d05d935f7416f89ea1873b0e42bb4` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:51:38.521938 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efba97fa-ee37-4fed-b951-56ac12ad876a None admin] RBAC: Authorizing `identity:get_user(user_id=f54d05d935f7416f89ea1873b0e42bb4)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:51:38.523738 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efba97fa-ee37-4fed-b951-56ac12ad876a None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:51:38.525396 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 598/1197] 10.222.0.27 () {56 vars in 1177 bytes} [Fri Aug 5 19:51:38 2022] GET /identity/v3/users/f54d05d935f7416f89ea1873b0e42bb4 => generated 311 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 05 19:51:38.531984 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a226a655-9b1b-4e99-895f-c893ef3f4c29 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:51:38.681779 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a226a655-9b1b-4e99-895f-c893ef3f4c29 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:51:38.719012 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57566e16-616a-4634-8f4c-8d9bcfc54580 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:51:38.719687 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57566e16-616a-4634-8f4c-8d9bcfc54580 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:51:38.720005 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57566e16-616a-4634-8f4c-8d9bcfc54580 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:51:38.720353 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57566e16-616a-4634-8f4c-8d9bcfc54580 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:51:38.749895 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:51:38.801544 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57566e16-616a-4634-8f4c-8d9bcfc54580 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:51:38.803085 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57566e16-616a-4634-8f4c-8d9bcfc54580 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:51:38.939794 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 600/1198] 10.222.0.27 () {58 vars in 1282 bytes} [Fri Aug 5 19:51:38 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 409 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 05 19:51:38.946017 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efba97fa-ee37-4fed-b951-56ac12ad876a None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:51:39.059655 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efba97fa-ee37-4fed-b951-56ac12ad876a None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:51:39.097637 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3026efbf-c49a-4c2e-8565-89bca4fec181 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:51:39.098521 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3026efbf-c49a-4c2e-8565-89bca4fec181 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:51:39.098849 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3026efbf-c49a-4c2e-8565-89bca4fec181 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:51:39.099329 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3026efbf-c49a-4c2e-8565-89bca4fec181 None admin] PATH_INFO: `/v3/projects/38e470bd429b46e091a36078bfbdea70` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:51:39.101222 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3026efbf-c49a-4c2e-8565-89bca4fec181 None admin] RBAC: Authorizing `identity:get_project(project_id=38e470bd429b46e091a36078bfbdea70)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:51:39.103316 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3026efbf-c49a-4c2e-8565-89bca4fec181 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:51:39.106021 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 599/1199] 10.222.0.27 () {56 vars in 1186 bytes} [Fri Aug 5 19:51:38 2022] GET /identity/v3/projects/38e470bd429b46e091a36078bfbdea70 => generated 361 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:51:39.114939 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57566e16-616a-4634-8f4c-8d9bcfc54580 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 05 19:51:39.190500 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:51:39.256319 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57566e16-616a-4634-8f4c-8d9bcfc54580 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 05 19:51:39.298242 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2e8fa3-4559-4284-a849-cb7139d8efdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d1834a64cd4a43829827cb99a72747', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 05 19:51:39.298886 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2e8fa3-4559-4284-a849-cb7139d8efdf None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 05 19:51:39.299166 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2e8fa3-4559-4284-a849-cb7139d8efdf None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 05 19:51:39.299487 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2e8fa3-4559-4284-a849-cb7139d8efdf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 05 19:51:39.300341 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2e8fa3-4559-4284-a849-cb7139d8efdf None admin] RBAC: Adding query filter params (role.id=c173688a960b46e5be65f0ccd6fbd734, scope.project.id=38e470bd429b46e091a36078bfbdea70, user.id=f54d05d935f7416f89ea1873b0e42bb4) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 05 19:51:39.300702 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2e8fa3-4559-4284-a849-cb7139d8efdf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 05 19:51:39.302212 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2e8fa3-4559-4284-a849-cb7139d8efdf None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 05 19:51:39.309579 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 601/1200] 10.222.0.27 () {56 vars in 1374 bytes} [Fri Aug 5 19:51:39 2022] GET /identity/v3/role_assignments?user.id=f54d05d935f7416f89ea1873b0e42bb4&scope.project.id=38e470bd429b46e091a36078bfbdea70&role.id=c173688a960b46e5be65f0ccd6fbd734 => generated 608 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 05 19:51:49.093131 dv-cinder-852208-2-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=74 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 05 19:51:49.306750 dv-cinder-852208-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=42 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}